Yep: http://prntscr.com/g3gab3
- - - - - - - - - - - - - - - - - - - Rodrigo Baldasso - LHOST (51) 9 8419-9861 - - - - - - - - - - - - - - - - - - - On 02/08/2017 11:02:40, Rafael Weingärtner <rafaelweingart...@gmail.com> wrote: I mean, I said 14 because this was the one select by you. Can you check the VLAN table searching by ID (vlan_db_id) from VLAN map table of the pod that you are having problems to remove. On Wed, Aug 2, 2017 at 11:00 AM, Rafael Weingärtner rafaelweingart...@gmail.com> wrote: > Interesting; now, can you try to find a vlan with ID=14 in "vlan" table? > > On Wed, Aug 2, 2017 at 10:58 AM, Rodrigo Baldasso > > wrote: > >> Sure: http://prntscr.com/g3g7iu >> >> The last vlan map is the one that I created for testing on the >> problematic pod. >> >> - - - - - - - - - - - - - - - - - - - >> >> Rodrigo Baldasso - LHOST >> >> (51) 9 8419-9861 >> - - - - - - - - - - - - - - - - - - - >> On 02/08/2017 10:56:01, Rafael Weingärtner >> wrote: >> Can you also list the table "pod_vlan_map" for all of the "maps" of vlans >> for the given POD? >> >> On Wed, Aug 2, 2017 at 10:40 AM, Rodrigo Baldasso >> wrote: >> >> > I've created a new vlan/ip pool just for this test (had already removed >> > all ip pool before), the problem persisted. >> > >> > here is how the new vlan looks like in my db: >> > >> > http://prntscr.com/g3fxt1 >> > >> > It's pretty equals to the one of the working pod. >> > >> > >> > >> > - - - - - - - - - - - - - - - - - - - >> > >> > Rodrigo Baldasso - LHOST >> > >> > (51) 9 8419-9861 >> > - - - - - - - - - - - - - - - - - - - >> > On 02/08/2017 10:21:47, Rafael Weingärtner >> > wrote: >> > Ok, >> > >> > Following the code where the null pointer happens at line 163: >> > https://github.com/apache/cloudstack/blob/0e057ad69edab9f166 >> 4924ac1fb250 >> > 0ca799cfb6/engine/schema/src/com/cloud/dc/dao/VlanDaoImpl.java >> > >> > can you list all VLANs in "vlan" table that have the PODid equals one of >> > the Pods you want to remove?Also, please filter the already removed >> VLANs. >> > >> > On Wed, Aug 2, 2017 at 9:26 AM, Rodrigo Baldasso >> > wrote: >> > >> > > This pod got somewhat corrupted when I change it's IP pool allocation >> > > cidr. After changing, it didn't started the SSVM's with the >> nullpointer >> > > error that i've send before and now the pod can't be deleted, also >> > throwing >> > > the nullpointer error. >> > > >> > > Thanks. >> > > >> > > - - - - - - - - - - - - - - - - - - - >> > > >> > > Rodrigo Baldasso - LHOST >> > > >> > > (51) 9 8419-9861 >> > > - - - - - - - - - - - - - - - - - - - >> > > On 02/08/2017 09:24:36, Rafael Weingärtner >> > > wrote: >> > > Well, looking at the data you posted, the problem should not happen. >> > Let´s >> > > take step back. Can you describe briefly what you are trying to do, >> and >> > > what problem you are having now? >> > > >> > > On Wed, Aug 2, 2017 at 9:17 AM, Rodrigo Baldasso >> > > wrote: >> > > >> > > > Hi Rafael, >> > > > >> > > > I checked this table yesterday and couldn't find nothing wrong.. i >> > made a >> > > > line around the vLan of the new pod (that is working). The other >> vLans >> > > are >> > > > from the pod that i'm unable to delete and didn't booted the SSVM's. >> > They >> > > > have the column 'removed' with the date that i deleted them. >> > > > >> > > > http://prntscr.com/g3eqpr >> > > > >> > > > - - - - - - - - - - - - - - - - - - - >> > > > >> > > > Rodrigo Baldasso - LHOST >> > > > >> > > > (51) 9 8419-9861 >> > > > - - - - - - - - - - - - - - - - - - - >> > > > On 02/08/2017 09:12:36, Rafael Weingärtner >> > > > wrote: >> > > > Rodrigo, >> > > > >> > > > There is an inconsistency in your database. >> > > > >> > > > To get this null pointer, the column "vlan_type" of table "vlan" >> must >> > be >> > > > null or something unrecognized in the database. Can you check this >> > table? >> > > > >> > > > On Tue, Aug 1, 2017 at 1:03 PM, Rodrigo Baldasso >> > > > wrote: >> > > > >> > > > > Thank you. >> > > > > >> > > > > This is the log when I try to delete this pod, if it helps. >> > > > > >> > > > > 2017-08-01 13:03:01,361 DEBUG [c.c.a.ApiServlet] >> > > > > (catalina-exec-9:ctx-262022ac) (logid:91c02e8f) ===START=== >> > > 186.250.14.20 >> > > > > -- GET command=deletePod&id=39e23530-0553-4b54-948d- >> > > > > 5da00c4952c2&response=json&_=1501603382823 >> > > > > 2017-08-01 13:03:01,556 DEBUG [c.c.u.d.T.Transaction] >> > > > > (catalina-exec-9:ctx-262022ac ctx-77de8cad) (logid:91c02e8f) >> Rolling >> > > back >> > > > > the transaction: Time = 179 Name = catalina-exec-9; called by >> > > > > -TransactionLegacy.rollback:889-TransactionLegacy.removeUpTo:832- >> > > > > TransactionLegacy.close:656-Transaction.execute:43- >> > > > Transaction.execute:47- >> > > > > ConfigurationManagerImpl.deletePod:1009-NativeMethodAccessorImpl. >> > > > > invoke0:-2-NativeMethodAccessorImpl.invoke:57- >> > > > > DelegatingMethodAccessorImpl.invoke:43-Method.invoke:606-Aop >> Utils. >> > > > > invokeJoinpointUsingReflection:317-ReflectiveMethodInvocation. >> > > > > invokeJoinpoint:183 >> > > > > 2017-08-01 13:03:01,667 ERROR [c.c.a.ApiServer] >> > > > > (catalina-exec-9:ctx-262022ac ctx-77de8cad) (logid:91c02e8f) >> > unhandled >> > > > > exception executing api command: [Ljava.lang.String;@1979ff2d >> > > > > java.lang.NullPointerException >> > > > > at com.cloud.dc.dao.VlanDaoImpl.listVlansForPodByType( >> > > > > VlanDaoImpl.java:163) >> > > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> > > > > at sun.reflect.NativeMethodAccessorImpl.invoke( >> > > > > NativeMethodAccessorImpl.java:57) >> > > > > at sun.reflect.DelegatingMethodAccessorImpl.invoke( >> > > > > DelegatingMethodAccessorImpl.java:43) >> > > > > at java.lang.reflect.Method.invoke(Method.java:606) >> > > > > at org.springframework.aop.support.AopUtils. >> > > > > invokeJoinpointUsingReflection(AopUtils.java:317) >> > > > > at org.springframework.aop.framework.ReflectiveMethodInvocation. >> > > > > invokeJoinpoint(ReflectiveMethodInvocation.java:183) >> > > > > at org.springframework.aop.framework.ReflectiveMethodInvocation. >> > > > > proceed(ReflectiveMethodInvocation.java:150) >> > > > > at com.cloud.utils.db.TransactionContextInterceptor.invoke( >> > > > > TransactionContextInterceptor.java:34) >> > > > > at org.springframework.aop.framework.ReflectiveMethodInvocation. >> > > > > proceed(ReflectiveMethodInvocation.java:161) >> > > > > at org.springframework.aop.interceptor. >> > > > > ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor. >> > > java:91) >> > > > > at org.springframework.aop.framework.ReflectiveMethodInvocation. >> > > > > proceed(ReflectiveMethodInvocation.java:172) >> > > > > at org.springframework.aop.framework.JdkDynamicAopProxy. >> > > > > invoke(JdkDynamicAopProxy.java:204) >> > > > > at com.sun.proxy.$Proxy113.listVlansForPodByType(Unknown Source) >> > > > > at com.cloud.network.NetworkModelImpl.listPodVlans( >> > > > > NetworkModelImpl.java:797) >> > > > > at com.cloud.configuration.ConfigurationManagerImpl$1. >> > > > > doInTransactionWithoutResult(ConfigurationManagerImpl.java:1029) >> > > > > at com.cloud.utils.db.TransactionCallbackNoReturn.doInTransacti >> on( >> > > > > TransactionCallbackNoReturn.java:25) >> > > > > at com.cloud.utils.db.Transaction$2.doInTransaction( >> > > > > Transaction.java:50) >> > > > > at com.cloud.utils.db.Transaction.execute(Transaction.java:40) >> > > > > at com.cloud.utils.db.Transaction.execute(Transaction.java:47) >> > > > > at com.cloud.configuration.ConfigurationManagerImpl.deletePod( >> > > > > ConfigurationManagerImpl.java:1009) >> > > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> > > > > at sun.reflect.NativeMethodAccessorImpl.invoke( >> > > > > NativeMethodAccessorImpl.java:57) >> > > > > at sun.reflect.DelegatingMethodAccessorImpl.invoke( >> > > > > DelegatingMethodAccessorImpl.java:43) >> > > > > at java.lang.reflect.Method.invoke(Method.java:606) >> > > > > at org.springframework.aop.support.AopUtils. >> > > > > invokeJoinpointUsingReflection(AopUtils.java:317) >> > > > > at org.springframework.aop.framework.ReflectiveMethodInvocation. >> > > > > invokeJoinpoint(ReflectiveMethodInvocation.java:183) >> > > > > at org.springframework.aop.framework.ReflectiveMethodInvocation. >> > > > > proceed(ReflectiveMethodInvocation.java:150) >> > > > > at org.springframework.aop.interceptor. >> > > > > ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor. >> > > java:91) >> > > > > at org.springframework.aop.framework.ReflectiveMethodInvocation. >> > > > > proceed(ReflectiveMethodInvocation.java:172) >> > > > > at org.springframework.aop.framework.JdkDynamicAopProxy. >> > > > > invoke(JdkDynamicAopProxy.java:204) >> > > > > at com.sun.proxy.$Proxy119.deletePod(Unknown Source) >> > > > > at org.apache.cloudstack.api.command.admin.pod. >> > > > > DeletePodCmd.execute(DeletePodCmd.java:70) >> > > > > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:150) >> > > > > at com.cloud.api.ApiServer.queueCommand(ApiServer.java:709) >> > > > > at com.cloud.api.ApiServer.handleRequest(ApiServer.java:533) >> > > > > at com.cloud.api.ApiServlet.processRequestInContext( >> > > > > ApiServlet.java:299) >> > > > > at com.cloud.api.ApiServlet$1.run(ApiServlet.java:129) >> > > > > at org.apache.cloudstack.managed.context.impl. >> > > > > DefaultManagedContext$1.call(DefaultManagedContext.java:56) >> > > > > at org.apache.cloudstack.managed.context.impl. >> > > > > DefaultManagedContext.callWithContext(DefaultManagedContext.java: >> > 103) >> > > > > at org.apache.cloudstack.managed.context.impl. >> > > > > DefaultManagedContext.runWithContext(DefaultManagedContext. >> java:53) >> > > > > at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:126) >> > > > > at com.cloud.api.ApiServlet.doGet(ApiServlet.java:88) >> > > > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:617) >> > > > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:723) >> > > > > at org.apache.catalina.core.ApplicationFilterChain. >> > > > > internalDoFilter(ApplicationFilterChain.java:290) >> > > > > at org.apache.catalina.core.ApplicationFilterChain.doFilter( >> > > > > ApplicationFilterChain.java:206) >> > > > > at org.apache.catalina.core.StandardWrapperValve.invoke( >> > > > > StandardWrapperValve.java:233) >> > > > > at org.apache.catalina.core.StandardContextValve.invoke( >> > > > > StandardContextValve.java:191) >> > > > > at org.apache.catalina.core.StandardHostValve.invoke( >> > > > > StandardHostValve.java:127) >> > > > > at org.apache.catalina.valves.ErrorReportValve.invoke( >> > > > > ErrorReportValve.java:102) >> > > > > at org.apache.catalina.valves.AccessLogValve.invoke( >> > > > > AccessLogValve.java:555) >> > > > > at org.apache.catalina.core.StandardEngineValve.invoke( >> > > > > StandardEngineValve.java:109) >> > > > > at org.apache.catalina.connector.CoyoteAdapter.service( >> > > > > CoyoteAdapter.java:299) >> > > > > at org.apache.coyote.http11.Http11NioProcessor.process( >> > > > > Http11NioProcessor.java:896) >> > > > > at org.apache.coyote.http11.Http11NioProtocol$ >> > > > > Http11ConnectionHandler.process(Http11NioProtocol.java:732) >> > > > > at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor. >> > > > > run(NioEndpoint.java:2277) >> > > > > at java.util.concurrent.ThreadPoolExecutor.runWorker( >> > > > > ThreadPoolExecutor.java:1145) >> > > > > at java.util.concurrent.ThreadPoolExecutor$Worker.run( >> > > > > ThreadPoolExecutor.java:615) >> > > > > at java.lang.Thread.run(Thread.java:748) >> > > > > >> > > > > >> > > > > - - - - - - - - - - - - - - - - - - - >> > > > > >> > > > > Rodrigo Baldasso - LHOST >> > > > > >> > > > > (51) 9 8419-9861 >> > > > > - - - - - - - - - - - - - - - - - - - >> > > > > On 01/08/2017 12:08:03, Gabriel Beims Bräscher >> > > > > wrote: >> > > > > It might be a bug then. >> > > > > >> > > > > I will try to connect the dots. The problem seems to be with the >> pod >> > > > then. >> > > > > >> > > > > 2017-08-01 12:01 GMT-03:00 Rodrigo Baldasso : >> > > > > >> > > > > > Hi, thanks for your reply >> > > > > > >> > > > > > I didnt made any changes manually on the database, however, it's >> > like >> > > > the >> > > > > > pod got corrupted after the change. I can't even delete the pod >> > from >> > > > the >> > > > > > zone that throws the exception (nullpointer) also. >> > > > > > >> > > > > > I was able to reproduce that in a fresh, clean install. So maybe >> > > it's a >> > > > > bug >> > > > > > or something related to my environment. >> > > > > > >> > > > > > Recreating the pod with the same parameters and settings works >> > fine. >> > > > > > >> > > > > > Enviado de GMail Mobile >> > > > > > >> > > > > > >> > > > > > Em 1 de ago de 2017 11:56, "Gabriel Beims Bräscher" >> > > > > > escreveu: >> > > > > > >> > > > > > You said that it worked before changing the IP range; it might >> be >> > > > caused >> > > > > by >> > > > > > a database inconsistency related to the IP allocation. Did you >> make >> > > any >> > > > > > manual changes in the database? >> > > > > > >> > > > > > The null pointer exception came after the following methods >> > > exectution: >> > > > > > 1 - it begins the exectution flow to start the VM >> > > > > > (com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart); >> > > > > > 2 - then it starts the network flow for that VM >> > > > > > (org.apache.cloudstack.engine.orchestration. >> > > > > NetworkOrchestrator.prepare); >> > > > > > 3 - it prepares the NIC >> > > > > > (org.apache.cloudstack.engine.orchestration.NetworkOrchestra >> tor. >> > > > > > prepareNic); >> > > > > > 4 - reserve the IP address >> > > > > > (com.cloud.network.guru.DirectPodBasedNetworkGuru.reserve); >> > > > > > 5 - finally it tries to get an IP address, triggering the null >> > > pointer >> > > > > > exception (com.cloud.network.guru.DirectPodBasedNetworkGuru. >> > getIp). >> > > > > > >> > > > > > 2017-07-31 13:27 GMT-03:00 Rodrigo Baldasso : >> > > > > > >> > > > > > > Thanks for your reply. >> > > > > > > >> > > > > > > I'm using 4.9.2.0. I'm posting the full log below! >> > > > > > > >> > > > > > > The secondary storage is mounted as NFS. >> > > > > > > >> > > > > > > PS: i was able to start SSVM after creating a new zone, >> however, >> > > > when I >> > > > > > > changed the IP range, the error came back again. >> > > > > > > >> > > > > > > 2017-07-31 13:23:15,950 DEBUG [c.c.a.t.Request] >> > > > > > > (Work-Job-Executor-6:ctx-e8c72967 job-131/job-272 >> ctx-06907041) >> > > > > > > (logid:7a78f881) Seq 1-6685312171854790673: Sending { Cmd , >> > MgmtId: >> > > > > > > 119779289367307, via: 1(kvm-tqr-01.lhost.net.br), Ver: v1, >> > Flags: >> > > > > > 100011, >> > > > > > > [{"com.cloud.agent.api.StopCommand":{"isProxy":false, >> > > > > > > "checkBeforeCleanup":false,"vmName":"v-66-VM"," >> > > > > > executeInSequence":false," >> > > > > > > wait":0}}] >> > > > > > > } >> > > > > > > 2017-07-31 13:23:16,153 DEBUG [c.c.a.t.Request] >> > > > > > > (AgentManager-Handler-7:null) (logid:) Seq >> 1-6685312171854790673: >> > > > > > > Processing: { Ans: , MgmtId: 119779289367307, via: 1, Ver: v1, >> > > Flags: >> > > > > > 10, >> > > > > > > [{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] >> } >> > > > > > > 2017-07-31 13:23:16,153 DEBUG [c.c.a.t.Request] >> > > > > > > (Work-Job-Executor-6:ctx-e8c72967 job-131/job-272 >> ctx-06907041) >> > > > > > > (logid:7a78f881) Seq 1-6685312171854790673: Received: { Ans: , >> > > > MgmtId: >> > > > > > > 119779289367307, via: 1(kvm-tqr-01.lhost.net.br), Ver: v1, >> > Flags: >> > > > 10, >> > > > > { >> > > > > > > StopAnswer } } >> > > > > > > 2017-07-31 13:23:16,166 DEBUG [c.c.n.NetworkModelImpl] >> > > > > > > (Work-Job-Executor-6:ctx-e8c72967 job-131/job-272 >> ctx-06907041) >> > > > > > > (logid:7a78f881) Service SecurityGroup is not supported in the >> > > > network >> > > > > > > id=204 >> > > > > > > 2017-07-31 13:23:16,201 DEBUG [o.a.c.e.o.NetworkOrchestrator] >> > > > > > > (Work-Job-Executor-6:ctx-e8c72967 job-131/job-272 >> ctx-06907041) >> > > > > > > (logid:7a78f881) Asking VirtualRouter to release >> > > > > > > NicProfile[196-66-cee1cba9-7a24-44cf-a5c2-24f2e5dff1ae-null- >> null >> > > > > > > 2017-07-31 13:23:16,211 DEBUG [c.c.v.VirtualMachineManagerIm >> pl] >> > > > > > > (Work-Job-Executor-6:ctx-e8c72967 job-131/job-272 >> ctx-06907041) >> > > > > > > (logid:7a78f881) Successfully released network resources for >> the >> > vm >> > > > > > > VM[ConsoleProxy|v-66-VM] >> > > > > > > 2017-07-31 13:23:16,211 DEBUG [c.c.v.VirtualMachineManagerIm >> pl] >> > > > > > > (Work-Job-Executor-6:ctx-e8c72967 job-131/job-272 >> ctx-06907041) >> > > > > > > (logid:7a78f881) Successfully cleanued up resources for the vm >> > > > > > > VM[ConsoleProxy|v-66-VM] in Starting state >> > > > > > > 2017-07-31 13:23:16,314 DEBUG [c.c.c.CapacityManagerImpl] >> > > > > > > (Work-Job-Executor-6:ctx-e8c72967 job-131/job-272 >> ctx-06907041) >> > > > > > > (logid:7a78f881) VM state transitted from :Starting to Stopped >> > with >> > > > > > event: >> > > > > > > OperationFailedvm's original host id: null new host id: null >> host >> > > id >> > > > > > before >> > > > > > > state transition: 1 >> > > > > > > 2017-07-31 13:23:16,324 DEBUG [c.c.c.CapacityManagerImpl] >> > > > > > > (Work-Job-Executor-6:ctx-e8c72967 job-131/job-272 >> ctx-06907041) >> > > > > > > (logid:7a78f881) Hosts's actual total CPU: 54432 and CPU after >> > > > applying >> > > > > > > overprovisioning: 54432 >> > > > > > > 2017-07-31 13:23:16,324 DEBUG [c.c.c.CapacityManagerImpl] >> > > > > > > (Work-Job-Executor-6:ctx-e8c72967 job-131/job-272 >> ctx-06907041) >> > > > > > > (logid:7a78f881) Hosts's actual total RAM: 7169134592 >> and RAM >> > after >> > > > > > > applying overprovisioning: 7169134592 >> > > > > > > 2017-07-31 13:23:16,325 DEBUG [c.c.c.CapacityManagerImpl] >> > > > > > > (Work-Job-Executor-6:ctx-e8c72967 job-131/job-272 >> ctx-06907041) >> > > > > > > (logid:7a78f881) release cpu from host: 1, old used: >> > 500,reserved: >> > > 0, >> > > > > > > actual total: 54432, total with overprovisioning: 54432; new >> > used: >> > > > > > > 0,reserved:0; movedfromreserved: false,moveToReserveredfalse >> > > > > > > 2017-07-31 13:23:16,325 DEBUG [c.c.c.CapacityManagerImpl] >> > > > > > > (Work-Job-Executor-6:ctx-e8c72967 job-131/job-272 >> ctx-06907041) >> > > > > > > (logid:7a78f881) release mem from host: 1, old used: >> > > > > 1073741824,reserved: >> > > > > > > 0, total: 7169134592 ; new used: >> 0,reserved:0; movedfromreserved: >> > > > > > > false,moveToReserveredfalse >> > > > > > > 2017-07-31 13:23:16,384 ERROR [c.c.v.VmWorkJobHandlerProxy] >> > > > > > > (Work-Job-Executor-6:ctx-e8c72967 job-131/job-272 >> ctx-06907041) >> > > > > > > (logid:7a78f881) Invocation exception, caused by: >> > > > > > > java.lang.NullPointerException >> > > > > > > 2017-07-31 13:23:16,384 INFO [c.c.v.VmWorkJobHandlerProxy] >> > > > > > > (Work-Job-Executor-6:ctx-e8c72967 job-131/job-272 >> ctx-06907041) >> > > > > > > (logid:7a78f881) Rethrow exception >> java.lang.NullPointerException >> > > > > > > 2017-07-31 13:23:16,384 DEBUG [c.c.v.VmWorkJobDispatcher] >> > > > > > > (Work-Job-Executor-6:ctx-e8c72967 job-131/job-272) >> > > (logid:7a78f881) >> > > > > Done >> > > > > > > with run of VM work job: com.cloud.vm.VmWorkStart for VM 66, >> job >> > > > > origin: >> > > > > > > 131 >> > > > > > > 2017-07-31 13:23:16,384 ERROR [c.c.v.VmWorkJobDispatcher] >> > > > > > > (Work-Job-Executor-6:ctx-e8c72967 job-131/job-272) >> > > (logid:7a78f881) >> > > > > > Unable >> > > > > > > to complete AsyncJobVO {id:272, userId: 1, accountId: 1, >> > > > instanceType: >> > > > > > > null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, >> cmdInfo: >> > > > > > > rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oA >> > > > > > > BGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50 >> > > > > > > UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5n >> > > > > > > L0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYX >> > > > > > > ZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB- >> > > > AAJMAAdwbGFubmVycQB- >> > > > > > > AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB- >> > AAJMAAlyYXdQYXJhbXN0AA9MamF2YS >> > > > > > > 91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB- >> > > AAN4cgATY29tLmNsb3VkLnZtLlZtV2 >> > > > > > 9ya5- >> > > > > > > ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmR >> > > > > > sZXJOYW1lcQB- >> > > > > > > AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAQnQAGVZpcnR1YWxNYWNoaW5l >> > > > > > > TWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA, >> > > > > > > cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, >> resultCode: >> > > 0, >> > > > > > > result: null, initMsid: 119779289367307, completeMsid: null, >> > > > > lastUpdated: >> > > > > > > null, lastPolled: null, created: Mon Jul 31 13:23:14 BRT >> 2017}, >> > job >> > > > > > > origin:131 >> > > > > > > java.lang.NullPointerException >> > > > > > > at >> > > > > > > com.cloud.network.guru.DirectPodBasedNetworkGuru$2. >> > > > > > > doInTransactionWithoutResult(DirectPodBasedNetworkGuru.java: >> 177) >> > > > > > > at >> > > > > > > com.cloud.utils.db.TransactionCallbackWithExcepti >> > > > > > > onNoReturn.doInTransaction(TransactionCallbackWithExcepti >> > > > > > > onNoReturn.java:25) >> > > > > > > at >> > > > > > > com.cloud.utils.db.TransactionCallbackWithExcepti >> > > > > > > onNoReturn.doInTransaction(TransactionCallbackWithExcepti >> > > > > > > onNoReturn.java:21) >> > > > > > > at com.cloud.utils.db.Transaction >> .execute(Transaction.java:40) >> > > > > > > at >> > > > > > > com.cloud.network.guru.DirectPodBasedNetworkGuru.getIp( >> > > > > > > DirectPodBasedNetworkGuru.java:170) >> > > > > > > at >> > > > > > > com.cloud.network.guru.DirectPodBasedNetworkGuru.reserve( >> > > > > > > DirectPodBasedNetworkGuru.java:157) >> > > > > > > at >> > > > > > > org.apache.cloudstack.engine.orchestration.NetworkOrchestrat >> or. >> > > > > > prepareNic( >> > > > > > > NetworkOrchestrator.java:1350) >> > > > > > > at >> > > > > > > org.apache.cloudstack.engine.orchestration. >> > > > > NetworkOrchestrator.prepare( >> > > > > > > NetworkOrchestrator.java:1321) >> > > > > > > at >> > > > > > > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart( >> > > > > > > VirtualMachineManagerImpl.java:997) >> > > > > > > at >> > > > > > > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart( >> > > > > > > VirtualMachineManagerImpl.java:4605) >> > > > > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native >> Method) >> > > > > > > at >> > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke( >> > > > > > NativeMethodAccessorImpl.java: >> > > > > > > 57) >> > > > > > > at >> > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke( >> > > > > > > DelegatingMethodAccessorImpl.java:43) >> > > > > > > at java.lang.reflect.Method.invoke(Method.java:606) >> > > > > > > at >> > > > > > > com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob( >> > > > > > > VmWorkJobHandlerProxy.java:107) >> > > > > > > at >> > > > > > > com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob( >> > > > > > > VirtualMachineManagerImpl.java:4766) >> > > > > > > at >> > > > > > > com.cloud.vm.VmWorkJobDispatcher.runJob( >> > > > VmWorkJobDispatcher.java:102) >> > > > > > > at >> > > > > > > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImp >> l$5. >> > > > > > > runInContext(AsyncJobManagerImpl.java:554) >> > > > > > > at >> > > > > > > org.apache.cloudstack.managed.context. >> > > ManagedContextRunnable$1.run( >> > > > > > > ManagedContextRunnable.java:49) >> > > > > > > at >> > > > > > > org.apache.cloudstack.managed.context.impl. >> > > > > DefaultManagedContext$1.call( >> > > > > > > DefaultManagedContext.java:56) >> > > > > > > at >> > > > > > > org.apache.cloudstack.managed.context.impl. >> > DefaultManagedContext. >> > > > > > > callWithContext(DefaultManagedContext.java:103) >> > > > > > > at >> > > > > > > org.apache.cloudstack.managed.context.impl. >> > DefaultManagedContext. >> > > > > > > runWithContext(DefaultManagedContext.java:53) >> > > > > > > at >> > > > > > > org.apache.cloudstack.managed.context. >> > ManagedContextRunnable.run( >> > > > > > > ManagedContextRunnable.java:46) >> > > > > > > at >> > > > > > > org.apache.cloudstack.framework.jobs.impl. >> > > AsyncJobManagerImpl$5.run( >> > > > > > > AsyncJobManagerImpl.java:502) >> > > > > > > at >> > > > > > > java.util.concurrent.Executors$RunnableAdapter. >> > > > > call(Executors.java:473) >> > > > > > > at java.util.concurrent.FutureTask.run(FutureTask.java:262) >> > > > > > > at >> > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker( >> > > > > > > ThreadPoolExecutor.java:1145) >> > > > > > > at >> > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run( >> > > > > > > ThreadPoolExecutor.java:615) >> > > > > > > at java.lang.Thread.run(Thread.java:748) >> > > > > > > 2017-07-31 13:23:16,388 DEBUG [o.a.c.f.j.i.AsyncJobManagerIm >> pl] >> > > > > > > (Work-Job-Executor-6:ctx-e8c72967 job-131/job-272) >> > > (logid:7a78f881) >> > > > > > > Complete async job-272, jobStatus: FAILED, resultCode: 0, >> result: >> > > > > > > rO0ABXNyAB5qYXZhLmxhbmcuTnVsbFBvaW50ZXJFeGNlcHRpb25HpaGO_ >> > > > > > > zHhuAIAAHhyABpqYXZhLmxhbmcuUnVudGltZUV4Y2VwdGlvbp5fBkcKNIPlA >> > > > > > > gAAeHIAE2phdmEubGFuZy5FeGNlcHRpb27Q_R8- >> > > > GjscxAIAAHhyABNqYXZhLmxhbmcuVG >> > > > > > > hyb3dhYmxl1cY1Jzl3uMsDAARMAAVjYXVzZXQAFUxqYXZhL2xhbmcvVGhyb3 >> > > > > > > dhYmxlO0wADWRldGFpbE1lc3NhZ2V0ABJMamF2YS9sYW5nL1N0cmluZztbAA >> > > > > > > pzdGFja1RyYWNldAAeW0xqYXZhL2xhbmcvU3RhY2tUcmFjZUVsZW1lbnQ7TA >> > > > > > > AUc3VwcHJlc3NlZEV4Y2VwdGlvbnN0ABBMamF2YS91dGlsL0xpc3Q7eHBxAH >> > > > > > > 4ACHB1cgAeW0xqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnQ7AkYqPDz9Ij >> > > > > > > kCAAB4cAAAAB1zcgAbamF2YS5sYW5nLlN0YWNrVHJhY2VFbGVtZW50YQnFmi >> > > > > > > Y23YUCAARJAApsaW5lTnVtYmVyTAAOZGVjbGFyaW5nQ2xhc3NxAH4ABUwACG >> > > > > > ZpbGVOYW1lcQB- >> > > > > > > AAVMAAptZXRob2ROYW1lcQB-AAV4cAAAALF0ADJjb20uY2xvdWQubm >> > > > > > > V0d29yay5ndXJ1LkRpcmVjdFBvZEJhc2VkTmV0d29ya0d1cnUkMnQAHkRpcm >> > > > > > > VjdFBvZEJhc2VkTmV0d29ya0d1cnUuamF2YXQAHGRvSW5UcmFuc2FjdGlvbl >> > > > > > > dpdGhvdXRSZXN1bHRzcQB-AAsAAAAZdAA7Y29tLmNsb3VkLnV0aW >> > > > > > > xzLmRiLlRyYW5zYWN0aW9uQ2FsbGJhY2tXaXRoRXhjZXB0aW9uTm9SZXR1cm >> > > > > > > 50AC1UcmFuc2FjdGlvbkNhbGxiYWNrV2l0aEV4Y2VwdGlvbk5vUmV0dXJuLm >> > > > > > > phdmF0AA9kb0luVHJhbnNhY3Rpb25zcQB-AAsAAAAVcQB- >> > > > > > > ABFxAH4AEnEAfgATc3EAfgALAAAAKHQAHmNvbS5jbG91ZC51dGlscy5kYi5U >> > > > > > > cmFuc2FjdGlvbnQAEFRyYW5zYWN0aW9uLmphdmF0AAdleGVjdXRlc3EAfgAL >> > > > > > > AAAAqnQAMGNvbS5jbG91ZC5uZXR3b3JrLmd1cnUuRGlyZWN0UG9kQmFzZWRO >> > > > > > > ZXR3b3JrR3VydXEAfgAOdAAFZ2V0SXBzcQB-AAsAAACdcQB- >> > > > > > > ABpxAH4ADnQAB3Jlc2VydmVzcQB-AAsAAAVGdAA- >> > > > b3JnLmFwYWNoZS5jbG91ZHN0YWNrLm >> > > > > > > VuZ2luZS5vcmNoZXN0cmF0aW9uLk5ldHdvcmtPcmNoZXN0cmF0b3J0ABhOZX >> > > > > > > R3b3JrT3JjaGVzdHJhdG9yLmphdmF0AApwcmVwYXJlTmljc3EAfgALAAAFKX >> > > > EAfgAfcQB- >> > > > > > > ACB0AAdwcmVwYXJlc3EAfgALAAAD5XQAJmNvbS5jbG91ZC52bS5WaXJ0dWFs >> > > > > > > TWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1w >> > > > > > > bC5qYXZhdAAQb3JjaGVzdHJhdGVTdGFydHNxAH4ACwAAEf1xAH4AJXEAfgAm >> > > > > > > cQB-ACdzcQB-AAv____-dAAkc3VuLnJlZmxlY3QuTmF0aXZlTW >> > > > > > > V0aG9kQWNjZXNzb3JJbXBsdAAdTmF0aXZlTWV0aG9kQWNjZXNzb3JJbXBsLm >> > > > > > > phdmF0AAdpbnZva2Uwc3EAfgALAAAAOXEAfgAqcQB-ACt0AAZpbnZva2VzcQ >> B- >> > > > > > > AAsAAAArdAAoc3VuLnJlZmxlY3QuRGVsZWdhdGluZ01ldGhvZEFjY2Vzc29y >> > > > > > > SW1wbHQAIURlbGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGwuamF2YXEAfgAu >> > > > > > > c3EAfgALAAACXnQAGGphdmEubGFuZy5yZWZsZWN0Lk1ldGhvZHQAC01ldGhv >> > > > > > > ZC5qYXZhcQB-AC5zcQB-AAsAAABrdAAiY29tLmNsb3VkLnZtLl >> > > > > > > ZtV29ya0pvYkhhbmRsZXJQcm94eXQAGlZtV29ya0pvYkhhbmRsZXJQcm94eS >> > > > > > > 5qYXZhdAAPaGFuZGxlVm1Xb3JrSm9ic3EAfgALAAASnnEAfgAlcQB- >> > > > > > > ACZxAH4AOHNxAH4ACwAAAGZ0ACBjb20uY2xvdWQudm0uVm1Xb3JrSm9iRGlz >> > > > > > > cGF0Y2hlcnQAGFZtV29ya0pvYkRpc3BhdGNoZXIuamF2YXQABnJ1bkpvYnNx >> > > > > > > AH4ACwAAAip0AD9vcmcuYXBhY2hlLmNsb3Vkc3RhY2suZnJhbWV3b3JrLmpv >> > > > > > > YnMuaW1wbC5Bc3luY0pvYk1hbmFnZXJJbXBsJDV0ABhBc3luY0pvYk1hbmFn >> > > > > > > ZXJJbXBsLmphdmF0AAxydW5JbkNvbnRleHRzcQB-AAsAAAAxdAA- >> > > > > > > b3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbmFnZWQuY29udGV4dC5NYW5hZ2Vk >> > > > > > > Q29udGV4dFJ1bm5hYmxlJDF0ABtNYW5hZ2VkQ29udGV4dFJ1bm5hYmxlLmph >> > > > > > > dmF0AANydW5zcQB-AAsAAAA4dABCb3JnLmFwYWNoZS5jbG >> > > > > > > 91ZHN0YWNrLm1hbmFnZWQuY29udGV4dC5pbXBsLkRlZmF1bHRNYW5hZ2VkQ2 >> > > > > > > 9udGV4dCQxdAAaRGVmYXVsdE1hbmFnZWRDb250ZXh0LmphdmF0AARjYWxsc3 >> > > > > > > EAfgALAAAAZ3QAQG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbn >> > > > > > > RleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHRxAH4ASHQAD2NhbGxXaX >> > > > > > > RoQ29udGV4dHNxAH4ACwAAADVxAH4AS3EAfgBIdAAOcnVuV2l0aENvbnRleH >> > RzcQB- >> > > > > > > AAsAAAAudAA8b3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbmFnZWQuY29udGV4 >> > > > > > > dC5NYW5hZ2VkQ29udGV4dFJ1bm5hYmxlcQB- >> > AERxAH4ARXNxAH4ACwAAAfZxAH4AP3 >> > > > > > > EAfgBAcQB-AEVzcQB-AAsAAAHZdAAuamF2YS51dGlsLmNvbm >> > > > > > > N1cnJlbnQuRXhlY3V0b3JzJFJ1bm5hYmxlQWRhcHRlcnQADkV4ZWN1dG9ycy >> > > > > > > 5qYXZhcQB-AElzcQB-AAsAAAEGdAAfamF2YS51dGlsLmNvbm >> > > > > > > N1cnJlbnQuRnV0dXJlVGFza3QAD0Z1dHVyZVRhc2suamF2YXEAfgBFc3EAfg >> > > > > > > ALAAAEeXQAJ2phdmEudXRpbC5jb25jdXJyZW50LlRocmVhZFBvb2xFeGVjdX >> > > > > > > RvcnQAF1RocmVhZFBvb2xFeGVjdXRvci5qYXZhdAAJcnVuV29ya2Vyc3EAfg >> > > > > > > ALAAACZ3QALmphdmEudXRpbC5jb25jdXJyZW50LlRocmVhZFBvb2xFeGVjdX >> > > > > > > RvciRXb3JrZXJxAH4AWnEAfgBFc3EAfgALAAAC7HQAEGphdmEubGFuZy5UaH >> > > > > > > JlYWR0AAtUaHJlYWQuamF2YXEAfgBFc3IAJmphdmEudXRpbC5Db2xsZWN0aW >> > > > > > > 9ucyRVbm1vZGlmaWFibGVMaXN0_A8lMbXsjhACAAFMAARsaXN0cQB- >> > > > > > > AAd4cgAsamF2YS51dGlsLkNvbGxlY3Rpb25zJFVubW9kaWZpYWJsZUNvbGxl >> > > > > > > Y3Rpb24ZQgCAy173HgIAAUwAAWN0ABZMamF2YS91dGlsL0NvbGxlY3Rpb247 >> > > > > > > eHBzcgATamF2YS51dGlsLkFycmF5TGlzdHiB0h2Zx2GdAwABSQAEc2l6ZXhw >> > > > > > > AAAAAHcEAAAAAHhxAH4AZng >> > > > > > > >> > > > > > > >> > > > > > > 2017-07-28 0:15 GMT-03:00 Gabriel Beims Bräscher : >> > > > > > > >> > > > > > > > Hi Rodrigo, >> > > > > > > > >> > > > > > > > Which version of CloudStack are you using? Is there any >> > exception >> > > > > stack >> > > > > > > in >> > > > > > > > the log? Are you using advanced or basic network? Is there >> any >> > > > > > connection >> > > > > > > > problem? >> > > > > > > > >> > > > > > > > Are you using Swift as Secondary Storage? If yes, there is >> an >> > > open >> > > > > > issue >> > > > > > > > "CLOUDSTACK-7443: Cannot launch SSVMs when using Swift as >> > > Secondary >> > > > > > > > Storage" (https://issues.apache.org/ >> > jira/browse/CLOUDSTACK-7443) >> > > > > > > > >> > > > > > > > Can you please review the log? Seems that there are missing >> > parts >> > > > > from >> > > > > > > the >> > > > > > > > presented log. One example is the line "2017-07-27 >> 22:30:00,253 >> > > > DEBUG >> > > > > > > > [c.c.c.CapacityManagerImpl]". The full log should contain >> "VM >> > > state >> > > > > > > > transitted from :" + oldState + " to " + newState + " with >> > event: >> > > > " + >> > > > > > > event >> > > > > > > > + "vm's original host id: " + vm.getLastHostId() + " new >> host >> > id: >> > > > " + >> > > > > > > > vm.getHostId() + " host id before state transition: " + >> > > oldHostId". >> > > > > > > > >> > > > > > > > Cheers, >> > > > > > > > Gabriel. >> > > > > > > > >> > > > > > > > 2017-07-27 22:37 GMT-03:00 Rodrigo Baldasso : >> > > > > > > > >> > > > > > > > > Hi everyone, >> > > > > > > > > >> > > > > > > > > I'm trying to set up my cloudstack environment but >> despite my >> > > > best >> > > > > > > > efforts, >> > > > > > > > > the system isn't starting the ssvm's.. on my agent log >> all i >> > > can >> > > > > see >> > > > > > > is: >> > > > > > > > > >> > > > > > > > > [kvm.resource.LibvirtConnection] >> > (agentRequest-Handler-5:null) >> > > > > > > > > (logid:4acd7d3e) Can not find a connection for Instance >> > > v-52-VM. >> > > > > > > Assuming >> > > > > > > > > the default connection. >> > > > > > > > > >> > > > > > > > > On the manager console, this is the error: >> > > > > > > > > >> > > > > > > > > 2017-07-27 22:29:59,853 DEBUG [c.c.a.t.Request] >> > > > > > > > > (Work-Job-Executor-54:ctx-a9fdb60b job-169/job-546 >> > > ctx-23fba537) >> > > > > > > > > (logid:4acd7d3e) Seq 1-2810527642455900231: >> > > > vm-tqr-01.lhost.net.br >> > > > > ), >> > > > > > > > Ver: >> > > > > > > > > v1, Flags: 100011, >> > > > > > > > > [{"com.cloud.agent.api.StopCommand":{"isProxy":false, >> > > > > > > > > "checkBeforeCleanup":false,"vmName":"v-52-VM","executeI >> > > > > > > > > 2017-07-27 22:29:59,952 DEBUG [o.a.c.f.j.i. >> > > AsyncJobManagerImpl] >> > > > > > > > > (secstorage-1:ctx-48b34715) (logid:526bc485) Sync job-547 >> > > > execution >> > > > > > on >> > > > > > > > > object VmWorkJobQueue. >> > > > > > > > > 2017-07-27 22:30:00,042 DEBUG [c.c.a.t.Request] >> > > > > > > > > (AgentManager-Handler-7:null) (logid:) Seq >> > > 1-2810527642455900231: >> > > > > > > > > Processing: { Ans: , MgmtId: >> > > > > > > > > 1197792893673.api.StopAnswer":{"result":true,"wait":0}}] >> } >> > > > > > > > > 2017-07-27 22:30:00,043 DEBUG [c.c.a.t.Request] >> > > > > > > > > (Work-Job-Executor-54:ctx-a9fdb60b job-169/job-546 >> > > ctx-23fba537) >> > > > > > > > > (logid:4acd7d3e) Seq 1-2810527642455900231: 1( >> > > > > > kvm-tqr-01.lhost.net.br >> > > > > > > ), >> > > > > > > > > Ver: v1, Flags: 10, { StopAnswer } } >> > > > > > > > > 2017-07-27 22:30:00,185 DEBUG [o.a.c.e.o. >> > NetworkOrchestrator] >> > > > > > > > > (Work-Job-Executor-54:ctx-a9fdb60b job-169/job-546 >> > > ctx-23fba537) >> > > > > > > > > (logid:4acd7d3e) Asking Virtua-42a5-a288-2582921a1946- >> > > null-null >> > > > > > > > > 2017-07-27 22:30:00,185 DEBUG [o.a.c.e.o. >> > NetworkOrchestrator] >> > > > > > > > > (Work-Job-Executor-54:ctx-a9fdb60b job-169/job-546 >> > > ctx-23fba537) >> > > > > > > > > (logid:4acd7d3e) Asking Securi129-658b-42a5-a288- >> > > > > > > 2582921a1946-null-null >> > > > > > > > > 2017-07-27 22:30:00,189 DEBUG [c.c.v. >> > > VirtualMachineManagerImpl] >> > > > > > > > > (Work-Job-Executor-54:ctx-a9fdb60b job-169/job-546 >> > > ctx-23fba537) >> > > > > > > > > (logid:4acd7d3e) SuccessfullProxy|v-52-VM] >> > > > > > > > > 2017-07-27 22:30:00,190 DEBUG [c.c.v. >> > > VirtualMachineManagerImpl] >> > > > > > > > > (Work-Job-Executor-54:ctx-a9fdb60b job-169/job-546 >> > > ctx-23fba537) >> > > > > > > > > (logid:4acd7d3e) Successfull|v-52-VM] in Starting state >> > > > > > > > > 2017-07-27 22:30:00,253 DEBUG [c.c.c.CapacityManagerImpl] >> > > > > > > > > (Work-Job-Executor-54:ctx-a9fdb60b job-169/job-546 >> > > ctx-23fba537) >> > > > > > > > > (logid:4acd7d3e) VM state transittFailedvm's original host >> > id: >> > > > null >> > > > > > new >> > > > > > > > > host id: null host id before state transition: 1 >> > > > > > > > > 2017-07-27 22:30:00,259 DEBUG [c.c.c.CapacityManagerImpl] >> > > > > > > > > (Work-Job-Executor-54:ctx-a9fdb60b job-169/job-546 >> > > ctx-23fba537) >> > > > > > > > > (logid:4acd7d3e) Hosts's actual toning: 54432 >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > >> > > > >> > > > -- >> > > > Rafael Weingärtner >> > > > >> > > >> > > >> > > >> > > -- >> > > Rafael Weingärtner >> > > >> > >> > >> > >> > -- >> > Rafael Weingärtner >> > >> >> >> >> -- >> Rafael Weingärtner >> > > > > -- > Rafael Weingärtner > -- Rafael Weingärtner
