[
https://issues.apache.org/jira/browse/CLOUDSTACK-7182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14113607#comment-14113607
]
Koushik Das commented on CLOUDSTACK-7182:
-----------------------------------------
Issue reported as part of parallel VM deployment tests, so that is already
known to QA.
Additionally network regression tests should also be run.
Testing done:
-----------------
1. Ran the network related tests under /test/integration/smoke folder using
Simulator. Also the other smoke tests didn’t cause any new failures.
2. Also ran the following VPC test using simulator:
a. created vpc, VR gets started as part of this
b. created network in the vpc
c. deployed vm in network created in step#2
3. Manually ran the following tests using simulator (for both regular VR and
VPC VR):
a. VR stop/start via API when the network is in Implemented state. Network
implement shouldn’t be triggered in this case.
Deployed VM, network gets implemented, stopped/started VR. Repeated this for
VPC as well.
b. VR start via API when network is Allocated state. It should trigger network
implement process. This test case is a good one to test the chicken-egg problem
– VR triggers network implement, and network implement in turn triggers the
provider implementation – which is the virtual router itself.
Deployed VM, network gets implemented, destroyed VM, after sometime VR gets
stopped and network goes to allocated state. At this point started the VR. Also
repeated same for VPC.
4. Ran #3 using XS setup as well
The following tests were also done by Alena as part of code review. These needs
to be executed as well and automated if not already done.
1. Update guest network when the network is a) implemented b) not implemented.
2. associate ip address list to account
3. start previously stopped router when the network is a) implemented b) not
implemented
4. create nic for vm from the network that is a) implement b) not implemented.
5. deploy multiple vms for the same network in parallel (tried for 4)
> NPE while trying to deploy VMs in parallel in isolated network
> --------------------------------------------------------------
>
> Key: CLOUDSTACK-7182
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-7182
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: Management Server
> Affects Versions: 4.4.0
> Reporter: Koushik Das
> Assignee: Koushik Das
> Labels: AUTOMATION_REQ, DEV_REVIEWED
> Fix For: 4.5.0
>
>
> Repro steps:
> - Create an account
> - Deploy around 50 VMs in parallel in a new network (VR not created)
> This is a race condition and NPE is not always hit. Below is the log snippet.
> 2014-03-06 12:20:15,121 DEBUG [c.c.h.x.r.CitrixResourceBase]
> (DirectAgent-233:ctx-210c1052) Creating VIF for i-4-98-VM on nic
> [Nic:Guest-10.1.1.105-null]
> 2014-03-06 12:20:15,127 WARN [c.c.h.x.r.CitrixResourceBase]
> (DirectAgent-233:ctx-210c1052) Catch Exception: class
> java.lang.NullPointerException due to java.lang.NullPointerException
> java.lang.NullPointerException
> at
> com.cloud.network.Networks$BroadcastDomainType.getSchemeValue(Networks.java:173)
> at
> com.cloud.network.Networks$BroadcastDomainType.getValue(Networks.java:228)
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNetwork(CitrixResourceBase.java:1084)
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.createVif(CitrixResourceBase.java:1137)
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1766)
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:627)
> at
> com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:60)
> at
> com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:106)
> at
> com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:215)
> at
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:50)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:47)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:701)
> 2014-03-06 12:20:15,128 WARN [c.c.h.x.r.CitrixResourceBase]
> (DirectAgent-233:ctx-210c1052) Unable to start i-4-98-VM due to
> java.lang.NullPointerException
> at
> com.cloud.network.Networks$BroadcastDomainType.getSchemeValue(Networks.java:173)
> at
> com.cloud.network.Networks$BroadcastDomainType.getValue(Networks.java:228)
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNetwork(CitrixResourceBase.java:1084)
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.createVif(CitrixResourceBase.java:1137)
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1766)
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:627)
> at
> com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:60)
> at
> com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:106)
> at
> com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:215)
> at
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:50)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:47)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:701)
> 2014-03-06 12:20:15,449 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-13:null) SeqA 3-10328: Processing Seq 3-10328: { Cmd ,
> MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2014-03-06 12:20:15,518 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-13:null) SeqA 3-10328: Sending Seq 3-10328: { Ans: ,
> MgmtId: 233845177509765, via: 3, Ver: v1, Flags: 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> 2014-03-06 12:20:15,561 WARN [c.c.h.x.r.CitrixResourceBase]
> (DirectAgent-233:ctx-210c1052) Unable to clean up VBD due to
> You gave an invalid object reference. The object may have recently been
> deleted. The class parameter gives the type of reference given, and the
> handle parameter echoes the bad value given.
> at com.xensource.xenapi.Types.checkResponse(Types.java:209)
> at com.xensource.xenapi.Connection.dispatch(Connection.java:368)
> at
> com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:469)
> at com.xensource.xenapi.VBD.unplug(VBD.java:1059)
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1510)
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1848)
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:627)
> at
> com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:60)
> at
> com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:106)
> at
> com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:215)
> at
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:50)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:47)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:701)
> 2014-03-06 12:20:15,564 WARN [c.c.h.x.r.CitrixResourceBase]
> (DirectAgent-233:ctx-210c1052) Unable to clean up VBD due to
> You gave an invalid object reference. The object may have recently been
> deleted. The class parameter gives the type of reference given, and the
> handle parameter echoes the bad value given.
> at com.xensource.xenapi.Types.checkResponse(Types.java:209)
> at com.xensource.xenapi.Connection.dispatch(Connection.java:368)
> at
> com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:469)
> at com.xensource.xenapi.VBD.unplug(VBD.java:1059)
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1510)
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1848)
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:627)
> at
> com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:60)
> at
> com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:106)
> at
> com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:215)
> at
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:50)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:47)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:701)
> 2014-03-06 12:20:15,565 DEBUG [c.c.h.x.r.CitrixResourceBase]
> (DirectAgent-233:ctx-210c1052) The VM is in stopped state, detected problem
> during startup : i-4-98-VM
> 2014-03-06 12:20:15,565 DEBUG [c.c.a.m.DirectAgentAttache]
> (DirectAgent-233:ctx-210c1052) Seq 2-195889156: Response Received:
>
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
