[ https://issues.apache.org/jira/browse/CLOUDSTACK-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devdeep Singh resolved CLOUDSTACK-1508. --------------------------------------- Resolution: Invalid Resolving the issue. Please retry after copying the tool and see if you are still facing the problem. > systems vms are fail to start > ----------------------------- > > Key: CLOUDSTACK-1508 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-1508 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: Management Server > Affects Versions: 4.2.0 > Reporter: sadhu suresh > Assignee: Devdeep Singh > Priority: Blocker > Attachments: management-server.rar > > > Hypervisor:xen6.0.2 > Once enabled the zone,Noticed below exception in the management log when > system VMs are in starting state and went to stopped state due to below > exceptions. > system vms are continuously failing with below exceptions. > Task failed! Task record: uuid: > d95cdf25-c7bb-ffe1-33f3-8f9324eddff2 > nameLabel: Async.VM.start_on > nameDescription: > allowedOperations: [] > currentOperations: {} > created: Fri Mar 01 11:05:09 EST 2013 > finished: Fri Mar 01 11:05:11 EST 2013 > status: failure > residentOn: com.xensource.xenapi.Host@31c6e197 > progress: 1.0 > type: <none/> > result: > errorInfo: [Traceback (most recent call last):, File > "/usr/bin/pygrub", line 808, in ?, fs = fsimage.open(file, part_offs[0], > bootfsoptions), IOError: [Errno 95] Operation not supported, ] > otherConfig: {} > subtaskOf: com.xensource.xenapi.Task@aaf13f6f > subtasks: [] > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.checkForSuccess(CitrixResourceBase.java:3283) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.startVM(CitrixResourceBase.java:3395) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1424) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:529) > at > com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73) > at > com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > 2013-03-01 10:58:47,636 WARN [xen.resource.CitrixResourceBase] > (DirectAgent-18:null) Catch Exception: class > com.cloud.utils.exception.CloudRuntimeException due to > com.cloud.utils.exception.CloudRuntimeException: Unable to start VM(s-14-VM) > on host(511ec001-1a19-400e-8e7e-11516d685475) due to Task failed! Task > record: uuid: d95cdf25-c7bb-ffe1-33f3-8f9324eddff2 > nameLabel: Async.VM.start_on > nameDescription: > allowedOperations: [] > currentOperations: {} > created: Fri Mar 01 11:05:09 EST 2013 > finished: Fri Mar 01 11:05:11 EST 2013 > status: failure > residentOn: com.xensource.xenapi.Host@31c6e197 > progress: 1.0 > type: <none/> > result: > errorInfo: [Traceback (most recent call last):, File > "/usr/bin/pygrub", line 808, in ?, fs = fsimage.open(file, part_offs[0], > bootfsoptions), IOError: [Errno 95] Operation not supported, ] > otherConfig: {} > subtaskOf: com.xensource.xenapi.Task@aaf13f6f > subtasks: [] > com.cloud.utils.exception.CloudRuntimeException: Unable to start VM(s-14-VM) > on host(511ec001-1a19-400e-8e7e-11516d685475) due to Task failed! Task > record: uuid: d95cdf25-c7bb-ffe1-33f3-8f9324eddff2 > nameLabel: Async.VM.start_on > nameDescription: > allowedOperations: [] > currentOperations: {} > created: Fri Mar 01 11:05:09 EST 2013 > finished: Fri Mar 01 11:05:11 EST 2013 > status: failure > residentOn: com.xensource.xenapi.Host@31c6e197 > progress: 1.0 > type: <none/> > result: > errorInfo: [Traceback (most recent call last):, File > "/usr/bin/pygrub", line 808, in ?, fs = fsimage.open(file, part_offs[0], > bootfsoptions), IOError: [Errno 95] Operation not supported, ] > otherConfig: {} > subtaskOf: com.xensource.xenapi.Task@aaf13f6f > subtasks: [] > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.startVM(CitrixResourceBase.java:3406) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1424) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:529) > at > com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73) > at > com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > 2013-03-01 10:58:47,637 WARN [xen.resource.CitrixResourceBase] > (DirectAgent-18:null) Unable to start s-14-VM due to > com.cloud.utils.exception.CloudRuntimeException: Unable to start VM(s-14-VM) > on host(511ec001-1a19-400e-8e7e-11516d685475) due to Task failed! Task > record: uuid: d95cdf25-c7bb-ffe1-33f3-8f9324eddff2 > nameLabel: Async.VM.start_on > nameDescription: > allowedOperations: [] > currentOperations: {} > created: Fri Mar 01 11:05:09 EST 2013 > finished: Fri Mar 01 11:05:11 EST 2013 > status: failure > residentOn: com.xensource.xenapi.Host@31c6e197 > progress: 1.0 > type: <none/> > result: > errorInfo: [Traceback (most recent call last):, File > "/usr/bin/pygrub", line 808, in ?, fs = fsimage.open(file, part_offs[0], > bootfsoptions), IOError: [Errno 95] Operation not supported, ] > otherConfig: {} > subtaskOf: com.xensource.xenapi.Task@aaf13f6f > subtasks: [] > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.startVM(CitrixResourceBase.java:3406) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1424) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:529) > at > com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73) > at > com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > 2013-03-01 10:58:47,684 DEBUG > [network.router.VirtualNetworkApplianceManagerImpl] > (RouterStatusMonitor-1:null) Found 0 routers to update status. > 2013-03-01 10:58:47,688 DEBUG > [network.router.VirtualNetworkApplianceManagerImpl] > (RouterStatusMonitor-1:null) Found 0 networks to update RvR status. > 2013-03-01 10:58:47,743 WARN [xen.resource.CitrixResourceBase] > (DirectAgent-18:null) Unable to clean up VBD due to You gave an invalid > object reference. The object may have recently been deleted. The class > parameter gives the type of reference given, and the handle parameter echoes > the bad value given. > at com.xensource.xenapi.Types.checkResponse(Types.java:209) > at com.xensource.xenapi.Connection.dispatch(Connection.java:368) > at > com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:909) > at com.xensource.xenapi.VBD.unplug(VBD.java:1059) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1224) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1487) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:529) > at > com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73) > at > com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > 2013-03-01 10:58:47,756 WARN [xen.resource.CitrixResourceBase] > (DirectAgent-18:null) Unable to clean up VBD due to You gave an invalid > object reference. The object may have recently been deleted. The class > parameter gives the type of reference given, and the handle parameter echoes > the bad value given. > at com.xensource.xenapi.Types.checkResponse(Types.java:209) > at com.xensource.xenapi.Connection.dispatch(Connection.java:368) > at > com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:909) > at com.xensource.xenapi.VBD.unplug(VBD.java:1059) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1224) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1487) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:529) > at > com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73) > at > com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > 2013-03-01 10:58:47,768 WARN [xen.resource.CitrixResourceBase] > (DirectAgent-18:null) Unable to cleanup VIF You gave an invalid object > reference. The object may have recently been deleted. The class parameter > gives the type of reference given, and the handle parameter echoes the bad > value given. > at com.xensource.xenapi.Types.checkResponse(Types.java:209) > at com.xensource.xenapi.Connection.dispatch(Connection.java:368) > at > com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:909) > at com.xensource.xenapi.VIF.unplug(VIF.java:878) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1232) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1487) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:529) > at > com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73) > at > com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > 2013-03-01 10:58:47,790 WARN [xen.resource.CitrixResourceBase] > (DirectAgent-18:null) Unable to cleanup VIF You gave an invalid object > reference. The object may have recently been deleted. The class parameter > gives the type of reference given, and the handle parameter echoes the bad > value given. > at com.xensource.xenapi.Types.checkResponse(Types.java:209) > at com.xensource.xenapi.Connection.dispatch(Connection.java:368) > at > com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:909) > at com.xensource.xenapi.VIF.unplug(VIF.java:878) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1232) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1487) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:529) > at > com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73) > at > com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > 2013-03-01 10:58:47,802 WARN [xen.resource.CitrixResourceBase] > (DirectAgent-18:null) Unable to cleanup VIF You gave an invalid object > reference. The object may have recently been deleted. The class parameter > gives the type of reference given, and the handle parameter echoes the bad > value given. > at com.xensource.xenapi.Types.checkResponse(Types.java:209) > at com.xensource.xenapi.Connection.dispatch(Connection.java:368) > at > com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:909) > at com.xensource.xenapi.VIF.unplug(VIF.java:878) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1232) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1487) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:529) > at > com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73) > at > com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > 2013-03-01 10:58:47,814 WARN [xen.resource.CitrixResourceBase] > (DirectAgent-18:null) Unable to cleanup VIF You gave an invalid object > reference. The object may have recently been deleted. The class parameter > gives the type of reference given, and the handle parameter echoes the bad > value given. > at com.xensource.xenapi.Types.checkResponse(Types.java:209) > at com.xensource.xenapi.Connection.dispatch(Connection.java:368) > at > com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:909) > at com.xensource.xenapi.VIF.unplug(VIF.java:878) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1232) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1487) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:529) > at > com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73) > at > com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > 2013-03-01 10:58:48,342 INFO [xen.resource.XenServer56Resource] > (DirectAgent-18:null) Catch com.xensource.xenapi.Types$InvalidValue: failed > to destory VLAN eth0 on host 511ec001-1a19-400e-8e7e-11516d685475 due to The > value given is invalid > 2013-03-01 10:58:48,344 DEBUG [xen.resource.CitrixResourceBase] > (DirectAgent-18:null) The VM is in stopped state, detected problem during > startup : s-14-VM > 2013-03-01 10:58:48,344 DEBUG [agent.manager.DirectAgentAttache] > (DirectAgent-18:null) Seq 1-1527644195: Cancelling because one of the answers > is false -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira