Hi Guys, Our setup is using VXLAN For Guest Networks.
We're trying to delete physical network but encountered this error. We've deleted all the instances, networks, system vms, hosts, pods and clusters and left with only the zone (disabled). Can someone please help tell us what we are missing? Regards, Bryan
Jan 11 11:38:39 N2NCLOUDMY1MG01 java[990006]: INFO [c.c.s.StatsCollector] (StatsCollector-1:ctx-ae5995fc) (logid:5b607178) system memory from /proc: 16744308736 Jan 11 11:38:39 N2NCLOUDMY1MG01 java[990006]: INFO [c.c.s.StatsCollector] (StatsCollector-1:ctx-ae5995fc) (logid:5b607178) free memory from /proc: 8095809536 Jan 11 11:38:39 N2NCLOUDMY1MG01 java[990006]: INFO [c.c.s.StatsCollector] (StatsCollector-1:ctx-ae5995fc) (logid:5b607178) used memory from /proc: 932188 Jan 11 11:38:39 N2NCLOUDMY1MG01 java[990006]: ERROR [c.c.c.ClusterServiceServletImpl] (Cluster-Worker-4:ctx-2e14bd30) (logid:6cad7a80) IOException from : http://10.21.1.38:9090/clusterservice, method : name=method, value=5 Jan 11 11:38:39 N2NCLOUDMY1MG01 java[990006]: ERROR [c.c.c.ClusterServiceServletImpl] (Cluster-Worker-4:ctx-2e14bd30) (logid:6cad7a80) IOException from : http://10.21.1.38:9090/clusterservice, method : name=method, value=5 Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: INFO [o.a.c.a.DynamicRoleBasedAPIAccessChecker] (qtp1550261631-290:ctx-39820611 ctx-55ae4289) (logid:9ceb8fa2) Account [Account [{"accountName":"admin","id":2,"uuid":"698819f7-9ef8-11ee-8ec2-5254007314cc"}]] is Root Admin or Domain Admin, all APIs are allowed. Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: WARN [o.a.c.a.ProjectRoleBasedApiAccessChecker] (qtp1550261631-290:ctx-39820611 ctx-55ae4289) (logid:9ceb8fa2) Project is null, ProjectRoleBasedApiAccessChecker only applies to projects, returning API [deletePhysicalNetwork] for user [User {"username":"admin","uuid":"6988be55-9ef8-11ee-8ec2-5254007314cc"}.] as allowed. Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: INFO [o.a.c.f.j.i.AsyncJobMonitor] (API-Job-Executor-34:ctx-54cc2479 job-6743) (logid:ad061e97) Add job-6743 into job monitoring Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-34:ctx-54cc2479 job-6743) (logid:1dfb6a24) Unexpected exception while executing org.apache.cloudstack.api.command.admin.network.DeletePhysicalNetworkCmd Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: com.cloud.utils.exception.CloudRuntimeException: The Physical Network is not deletable because there are allocated vnets for this physical network Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at com.cloud.network.NetworkServiceImpl.checkIfPhysicalNetworkIsDeletable(NetworkServiceImpl.java:4387) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at com.cloud.network.NetworkServiceImpl.deletePhysicalNetwork(NetworkServiceImpl.java:4253) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at jdk.internal.reflect.GeneratedMethodAccessor355.invoke(Unknown Source) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at java.base/java.lang.reflect.Method.invoke(Method.java:566) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:52) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at com.sun.proxy.$Proxy278.deletePhysicalNetwork(Unknown Source) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at org.apache.cloudstack.api.command.admin.network.DeletePhysicalNetworkCmd.execute(DeletePhysicalNetworkCmd.java:71) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:163) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:112) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: at java.base/java.lang.Thread.run(Thread.java:829) Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: INFO [o.a.c.f.j.i.AsyncJobMonitor] (API-Job-Executor-34:ctx-54cc2479 job-6743) (logid:1dfb6a24) Remove job-6743 from job monitoring Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: INFO [o.a.c.a.DynamicRoleBasedAPIAccessChecker] (qtp1550261631-348:ctx-1293ad97 ctx-415e673f) (logid:b3c141c8) Account [Account [{"accountName":"admin","id":2,"uuid":"698819f7-9ef8-11ee-8ec2-5254007314cc"}]] is Root Admin or Domain Admin, all APIs are allowed. Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: WARN [o.a.c.a.ProjectRoleBasedApiAccessChecker] (qtp1550261631-348:ctx-1293ad97 ctx-415e673f) (logid:b3c141c8) Project is null, ProjectRoleBasedApiAccessChecker only applies to projects, returning API [queryAsyncJobResult] for user [User {"username":"admin","uuid":"6988be55-9ef8-11ee-8ec2-5254007314cc"}.] as allowed. Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: INFO [o.a.c.a.DynamicRoleBasedAPIAccessChecker] (qtp1550261631-21:ctx-e75ecdf6 ctx-e6874f7c) (logid:8d643db9) Account [Account [{"accountName":"admin","id":2,"uuid":"698819f7-9ef8-11ee-8ec2-5254007314cc"}]] is Root Admin or Domain Admin, all APIs are allowed. Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: WARN [o.a.c.a.ProjectRoleBasedApiAccessChecker] (qtp1550261631-21:ctx-e75ecdf6 ctx-e6874f7c) (logid:8d643db9) Project is null, ProjectRoleBasedApiAccessChecker only applies to projects, returning API [listResourceIcon] for user [User {"username":"admin","uuid":"6988be55-9ef8-11ee-8ec2-5254007314cc"}.] as allowed. Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: INFO [o.a.c.a.DynamicRoleBasedAPIAccessChecker] (qtp1550261631-290:ctx-13363f34 ctx-ee68e72a) (logid:bc05d1b2) Account [Account [{"accountName":"admin","id":2,"uuid":"698819f7-9ef8-11ee-8ec2-5254007314cc"}]] is Root Admin or Domain Admin, all APIs are allowed. Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: WARN [o.a.c.a.ProjectRoleBasedApiAccessChecker] (qtp1550261631-290:ctx-13363f34 ctx-ee68e72a) (logid:bc05d1b2) Project is null, ProjectRoleBasedApiAccessChecker only applies to projects, returning API [listZonesMetrics] for user [User {"username":"admin","uuid":"6988be55-9ef8-11ee-8ec2-5254007314cc"}.] as allowed. Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: WARN [c.c.a.d.ParamGenericValidationWorker] (qtp1550261631-290:ctx-13363f34 ctx-ee68e72a) (logid:bc05d1b2) Received unknown parameters for command listZonesMetrics. Unknown parameters : projectid Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: INFO [o.a.c.a.DynamicRoleBasedAPIAccessChecker] (qtp1550261631-22:ctx-447d8cbf ctx-ae2a85eb) (logid:87cd2297) Account [Account [{"accountName":"admin","id":2,"uuid":"698819f7-9ef8-11ee-8ec2-5254007314cc"}]] is Root Admin or Domain Admin, all APIs are allowed. Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: INFO [o.a.c.a.DynamicRoleBasedAPIAccessChecker] (qtp1550261631-348:ctx-85cd1e9e ctx-377f40b6) (logid:bfa33ea1) Account [Account [{"accountName":"admin","id":2,"uuid":"698819f7-9ef8-11ee-8ec2-5254007314cc"}]] is Root Admin or Domain Admin, all APIs are allowed. Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: WARN [o.a.c.a.ProjectRoleBasedApiAccessChecker] (qtp1550261631-22:ctx-447d8cbf ctx-ae2a85eb) (logid:87cd2297) Project is null, ProjectRoleBasedApiAccessChecker only applies to projects, returning API [listVmwareDcs] for user [User {"username":"admin","uuid":"6988be55-9ef8-11ee-8ec2-5254007314cc"}.] as allowed. Jan 11 11:38:41 N2NCLOUDMY1MG01 java[990006]: WARN [o.a.c.a.ProjectRoleBasedApiAccessChecker] (qtp1550261631-348:ctx-85cd1e9e ctx-377f40b6) (logid:bfa33ea1) Project is null, ProjectRoleBasedApiAccessChecker only applies to projects, returning API [listDedicatedZones] for user [User {"username":"admin","uuid":"6988be55-9ef8-11ee-8ec2-5254007314cc"}.] as allowed.
