Yes, Agent running and connected to management. Agent can provision new VM's but can't do something with old running instances ( volumes ) After upgrade I can not restart or backup the instance.
NFS running and rpcbind running properly ( NFS re-mounted before upgrade ) [root@kvmhost2 ~]# /etc/init.d/nfs status rpc.svcgssd is stopped rpc.mountd (pid 29424) is running... nfsd (pid 29421 29420 29419 29418 29417 29416 29415 29414) is running... rpc.rquotad (pid 29408) is running... [root@ kvmhost2 ~]# mount | grep nfs sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) nfsd on /proc/fs/nfsd type nfsd (rw) NFSHOST:/opt/secondary on /mnt/7cd8eb7b-9d44-33e4-8e30-3a3f2150d9c0 type nfs (rw,noac,vers=4,addr=x.x.x.x..x.x,clientaddr=x.x.x.x.x) 2012-06-03 21:20:03,222 WARN [resource.computing.LibvirtComputingResource] (agentRequest-Handler-4:null) Exception com.cloud.utils.exception.CloudRuntimeException: org.libvirt.LibvirtException: Storage volume not found: no storage vol with matching name '/var/lib/libvirt/images/19d086dd-8229-48c2-aaa7-ea3d218dc5dc' at com.cloud.agent.storage.LibvirtStorageAdaptor.getVolume(LibvirtStorageAdaptor.java:81) at com.cloud.agent.storage.LibvirtStorageAdaptor.getPhysicalDisk(LibvirtStorageAdaptor.java:450) at com.cloud.agent.storage.LibvirtStoragePool.getPhysicalDisk(LibvirtStoragePool.java:104) at com.cloud.agent.resource.computing.LibvirtComputingResource.createVbd(LibvirtComputingResource.java:2593) at com.cloud.agent.resource.computing.LibvirtComputingResource.execute(LibvirtComputingResource.java:2510) at com.cloud.agent.resource.computing.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:985) at com.cloud.agent.Agent.processRequest(Agent.java:517) at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:827) at com.cloud.utils.nio.Task.run(Task.java:79) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679) 2012-06-03 21:20:03,224{GMT} WARN [cloud.agent.Agent] (agentRequest-Handler-4:) Caught: java.lang.NullPointerException at com.cloud.agent.resource.computing.LibvirtComputingResource.cleanupVMNetworks(LibvirtComputingResource.java:3664) at com.cloud.agent.resource.computing.LibvirtComputingResource.handleVmStartFailure(LibvirtComputingResource.java:2416) at com.cloud.agent.resource.computing.LibvirtComputingResource.execute(LibvirtComputingResource.java:2539) at com.cloud.agent.resource.computing.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:985) at com.cloud.agent.Agent.processRequest(Agent.java:517) at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:827) at com.cloud.utils.nio.Task.run(Task.java:79) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679) -- Serg. On Jun 3, 2012, at 11:18 PM, Chandan Purushothama wrote: > Hello Serg, > > Is the Cloud-Agent service running on the KVM host after Upgrade? Are the nfs > and rpcbind services on the KVM host running after the Upgrade? > > Thank you, > Chandan. > > -----Original Message----- > From: Serg [mailto:kernc...@gmail.com] > Sent: Sunday, June 03, 2012 8:15 AM > To: cloudstack-users@incubator.apache.org > Subject: Re: Snapshot failed after upgrade to 3.0.2 from 2.2.14 ( KVM ) > > Same problem with cleanup deleted volumes : > > 2012-06-03 17:07:48,911 DEBUG [agent.transport.Request] > (AgentManager-Handler-14:null) Seq 38-259787363: Processing: { Ans: , > MgmtId: 207382744522, via: 38, Ver: v1, Flags: 110, > [{"Answer":{"result":false,"details":"com.cloud.utils.exception.CloudRuntimeException: > org.libvirt.LibvirtException: Storage volume not found: no storage vol with > matching name > '/var/lib/libvirt/images/a1b259d0-ac91-4446-9b43-e19d2327454b'","wait":0}}] } > > Volume exist on local storage. > > -- > Serg. > > > On Jun 3, 2012, at 6:05 PM, Serg wrote: > >> Hi, >> >> Local storage used. >> NFS used as secondary storage. >> >> After upgrade from 2.2.14 to 3.0.2 snapshots has failed with the following >> error : >> >> Agent log : >> >> com.cloud.utils.exception.CloudRuntimeException: >> org.libvirt.LibvirtException: Storage volume not found: no storage vol with >> matching name '/var/lib/libvirt/images/e4834e2b-884b-4f6d-97a6-eae509d644e3' >> at >> com.cloud.agent.storage.LibvirtStorageAdaptor.getVolume(LibvirtStorageAdaptor.java:81) >> at >> com.cloud.agent.storage.LibvirtStorageAdaptor.getPhysicalDisk(LibvirtStorageAdaptor.java:450) >> at >> com.cloud.agent.storage.LibvirtStoragePool.getPhysicalDisk(LibvirtStoragePool.java:104) >> at >> com.cloud.agent.resource.computing.LibvirtComputingResource.execute(LibvirtComputingResource.java:1284) >> at >> com.cloud.agent.resource.computing.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:965) >> at com.cloud.agent.Agent.processRequest(Agent.java:517) >> at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:827) >> at com.cloud.utils.nio.Task.run(Task.java:79) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) >> at java.lang.Thread.run(Thread.java:679) >> >> >> Volume exist and instance running : >> >> [root@kvmhost2 /]# ls -al >> /var/lib/libvirt/images/e4834e2b-884b-4f6d-97a6-eae509d644e3 >> -rw-------. 1 root root 2664431616 Jun 3 16:54 >> /var/lib/libvirt/images/e4834e2b-884b-4f6d-97a6-eae509d644e3 >> >> >> >> >