Ok, that looks like an issue. Can you please report it if not existing already on: https://github.com/apache/cloudstack/issues with the steps you have done to reproduce it?
Regards, Nicolas Vazquez ________________________________ From: Hean Seng <heans...@gmail.com> Sent: Wednesday, April 14, 2021 11:49 AM To: users@cloudstack.apache.org <users@cloudstack.apache.org> Subject: Re: Request failed with status code 530 Hi I think this is really a bug, each host suppose have one local storage only, but it return two, after digging to storage_pool , it become NULL in uuid , after i remove this record, then listHosts accessible now. select * from storage_pool_host_ref where host_id=17; +----+---------+---------+---------------------+--------------+-------------------------------------------+ | id | host_id | pool_id | created | last_updated | local_path | +----+---------+---------+---------------------+--------------+-------------------------------------------+ | 27 | 17 | 3 | 2021-04-12 18:45:27 | NULL | /mnt/1f45642b-f8f3-349f-8e46-8f3eb983da23 | | 28 | 17 | 14 | 2021-04-12 18:45:27 | NULL | /var/lib/libvirt/images | | 29 | 17 | 15 | 2021-04-12 18:51:50 | NULL | /var/lib/libvirt/images | +----+---------+---------+---------------------+--------------+-------------------------------------------+ mysql> select id, uuid, name, status from storage_pool where id=14; +----+------+---------------------------------+-------------+ | id | uuid | name | status | +----+------+---------------------------------+-------------+ | 14 | NULL | lax-06 | Maintenance | +----+------+---------------------------------+-------------+ 1 row in set (0.00 sec) mysql> select id, uuid, name, status from storage_pool where id=15; +----+--------------------------------------+---------------------------------+-------------+ | id | uuid | name | status | +----+--------------------------------------+---------------------------------+-------------+ | 15 | 30717adf-46ba-43fe-bac9-038ef6039fda | lax-06 | Maintenance | +----+--------------------------------------+---------------------------------+-------------+ 1 row in set (0.00 sec) On Wed, Apr 14, 2021 at 10:17 PM Nicolas Vazquez < nicolas.vazq...@shapeblue.com> wrote: > Thanks, > > Have you deleted any storage pool accesible to the host? Please check if > there are not missing references on the table 'storage_pool_host_ref' for > host_id = 17. It could also be the case that a record on that table is > pointing to a removed storage pool > > Can you try putting the host into maintenance first before deleting it? > > > Regards, > > Nicolas Vazquez > > ________________________________ > From: Hean Seng <heans...@gmail.com> > Sent: Wednesday, April 14, 2021 10:30 AM > To: users@cloudstack.apache.org <users@cloudstack.apache.org> > Subject: Re: Request failed with status code 530 > > Hi Nicolas, > > Thanks for reply , following is error : > >>> # # cmk listHosts id=17 > > 2021-04-14 13:26:47,391 DEBUG [c.c.a.ApiServer] > (qtp1644231115-2156:ctx-47a63ef7 ctx-6c8f0fae) (logid:f5043840) CIDRs from > which account 'Acct[b10cca30-6fb6-11eb-b8cc-ce70b13735fa-admin]' is allowed > to perform API calls: 0.0.0.0/0,::/0 > > 2021-04-14 13:26:47,396 DEBUG [c.c.a.q.QueryManagerImpl] > (qtp1644231115-2156:ctx-47a63ef7 ctx-6c8f0fae) (logid:f5043840) > >>>Searching for hosts>>> > > 2021-04-14 13:26:47,402 DEBUG [c.c.a.q.QueryManagerImpl] > (qtp1644231115-2156:ctx-47a63ef7 ctx-6c8f0fae) (logid:f5043840) > >>>Generating Response>>> > > 2021-04-14 13:26:47,408 ERROR [c.c.a.ApiServer] > (qtp1644231115-2156:ctx-47a63ef7 ctx-6c8f0fae) (logid:f5043840) unhandled > exception executing api command: [Ljava.lang.String;@34315235 > > java.lang.NullPointerException > > at > > com.cloud.storage.StorageManagerImpl.isLocalStorageActiveOnHost(StorageManagerImpl.java:404) > > at jdk.internal.reflect.GeneratedMethodAccessor372.invoke(Unknown Source) > > at > > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.base/java.lang.reflect.Method.invoke(Method.java:566) > > at > > org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) > > at > > org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) > > at > > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) > > at > > org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) > > at > > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) > > at > > org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) > > at com.sun.proxy.$Proxy96.isLocalStorageActiveOnHost(Unknown Source) > > at com.cloud.api.ApiDBUtils.isLocalStorageActiveOnHost(ApiDBUtils.java:955) > > at > > com.cloud.api.query.dao.HostJoinDaoImpl.newHostResponse(HostJoinDaoImpl.java:238) > > at jdk.internal.reflect.GeneratedMethodAccessor370.invoke(Unknown Source) > > at > > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.base/java.lang.reflect.Method.invoke(Method.java:566) > > at > > org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) > > at > > org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) > > at > > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) > > at > > com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34) > > at > > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) > > at > > org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) > > at > > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) > > at > > org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) > > at com.sun.proxy.$Proxy316.newHostResponse(Unknown Source) > > at com.cloud.api.ApiDBUtils.newHostResponse(ApiDBUtils.java:1870) > > at > > com.cloud.api.query.ViewResponseHelper.createHostResponse(ViewResponseHelper.java:248) > > at > > com.cloud.api.query.QueryManagerImpl.searchForServers(QueryManagerImpl.java:1748) > > at > > org.apache.cloudstack.api.command.admin.host.ListHostsCmd.getHostResponses(ListHostsCmd.java:208) > > at > > org.apache.cloudstack.api.command.admin.host.ListHostsCmd.execute(ListHostsCmd.java:233) > > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156) > > at com.cloud.api.ApiServer.queueCommand(ApiServer.java:764) > > at com.cloud.api.ApiServer.handleRequest(ApiServer.java:588) > > at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:321) > > at com.cloud.api.ApiServlet$1.run(ApiServlet.java:134) > > at > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55) > > at > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102) > > at > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52) > > at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:131) > > at com.cloud.api.ApiServlet.doGet(ApiServlet.java:93) > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:645) > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:750) > > at > > org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1386) > > at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:755) > > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:547) > > at > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) > > at > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > > at > > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) > > at > > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) > > at > > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) > > at > > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300) > > at > > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) > > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485) > > at > > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) > > at > > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) > > at > > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215) > > at > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) > > at > > org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:767) > > at > > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) > > at > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > > at org.eclipse.jetty.server.Server.handle(Server.java:500) > > at > org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) > > at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547) > > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) > > at > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) > > at > org.eclipse.jetty.io > .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) > > at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) > > at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) > > at > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) > > at > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) > > at > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) > > at > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) > > at > > org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) > > at > > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) > > at > > org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) > > at java.base/java.lang.Thread.run(Thread.java:834) > > > > > >>> cmk deletehost id=17 forced=true > > > > 2021-04-14 13:29:21,967 DEBUG [c.c.r.ResourceManagerImpl] > (qtp1644231115-2157:ctx-6167b69c ctx-f0d83d44) (logid:11607e2f) Cannot > transmit host 17 to Enabled state > > com.cloud.utils.fsm.NoTransitionException: No next resource state found for > current state = Enabled event = DeleteHost > > at > > com.cloud.resource.ResourceManagerImpl.resourceStateTransitTo(ResourceManagerImpl.java:1213) > > at > > com.cloud.resource.ResourceManagerImpl$1.doInTransactionWithoutResult(ResourceManagerImpl.java:904) > > at > > com.cloud.utils.db.TransactionCallbackNoReturn.doInTransaction(TransactionCallbackNoReturn.java:25) > > at com.cloud.utils.db.Transaction$2.doInTransaction(Transaction.java:50) > > at com.cloud.utils.db.Transaction.execute(Transaction.java:40) > > at com.cloud.utils.db.Transaction.execute(Transaction.java:47) > > at > > com.cloud.resource.ResourceManagerImpl.doDeleteHost(ResourceManagerImpl.java:873) > > at > > com.cloud.resource.ResourceManagerImpl.deleteHost(ResourceManagerImpl.java:967) > > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > > at > > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > at > > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.base/java.lang.reflect.Method.invoke(Method.java:566) > > at > > org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) > > at > > org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) > > at > > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) > > at > > org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) > > at > > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) > > at > > org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) > > at com.sun.proxy.$Proxy188.deleteHost(Unknown Source) > > at > > org.apache.cloudstack.api.command.admin.host.DeleteHostCmd.execute(DeleteHostCmd.java:88) > > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156) > > at com.cloud.api.ApiServer.queueCommand(ApiServer.java:764) > > at com.cloud.api.ApiServer.handleRequest(ApiServer.java:588) > > at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:321) > > at com.cloud.api.ApiServlet$1.run(ApiServlet.java:134) > > at > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55) > > at > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102) > > at > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52) > > at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:131) > > at com.cloud.api.ApiServlet.doGet(ApiServlet.java:93) > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:645) > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:750) > > at > > org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1386) > > at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:755) > > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:547) > > at > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) > > at > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > > at > > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) > > at > > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) > > at > > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) > > at > > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300) > > at > > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) > > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485) > > at > > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) > > at > > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) > > at > > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215) > > at > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) > > at > > org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:767) > > at > > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) > > at > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > > at org.eclipse.jetty.server.Server.handle(Server.java:500) > > at > org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) > > at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547) > > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) > > at > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) > > at > org.eclipse.jetty.io > .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) > > at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) > > at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) > > at > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) > > at > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) > > at > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) > > at > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) > > at > > org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) > > at > > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) > > at > > org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) > > at java.base/java.lang.Thread.run(Thread.java:834) > > 2021-04-14 13:29:21,971 DEBUG [c.c.u.d.T.Transaction] > (qtp1644231115-2157:ctx-6167b69c ctx-f0d83d44) (logid:11607e2f) Rolling > back the transaction: Time = 19 Name = qtp1644231115-2157; called by > > -TransactionLegacy.rollback:888-TransactionLegacy.removeUpTo:831-TransactionLegacy.close:655-Transaction.execute:38-Transaction.execute:47-ResourceManagerImpl.doDeleteHost:873-ResourceManagerImpl.deleteHost:967-NativeMethodAccessorImpl.invoke0:-2-NativeMethodAccessorImpl.invoke:62-DelegatingMethodAccessorImpl.invoke:43-Method.invoke:566-AopUtils.invokeJoinpointUsingReflection:344 > > 2021-04-14 13:29:21,974 ERROR [c.c.a.ApiServer] > (qtp1644231115-2157:ctx-6167b69c ctx-f0d83d44) (logid:11607e2f) unhandled > exception executing api command: [Ljava.lang.String;@38e732da > > java.lang.NullPointerException > > at > > com.cloud.resource.ResourceManagerImpl$1.doInTransactionWithoutResult(ResourceManagerImpl.java:926) > > at > > com.cloud.utils.db.TransactionCallbackNoReturn.doInTransaction(TransactionCallbackNoReturn.java:25) > > at com.cloud.utils.db.Transaction$2.doInTransaction(Transaction.java:50) > > at com.cloud.utils.db.Transaction.execute(Transaction.java:40) > > at com.cloud.utils.db.Transaction.execute(Transaction.java:47) > > at > > com.cloud.resource.ResourceManagerImpl.doDeleteHost(ResourceManagerImpl.java:873) > > at > > com.cloud.resource.ResourceManagerImpl.deleteHost(ResourceManagerImpl.java:967) > > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > > at > > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > at > > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.base/java.lang.reflect.Method.invoke(Method.java:566) > > at > > org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) > > at > > org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) > > at > > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) > > at > > org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) > > at > > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) > > at > > org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) > > at com.sun.proxy.$Proxy188.deleteHost(Unknown Source) > > at > > org.apache.cloudstack.api.command.admin.host.DeleteHostCmd.execute(DeleteHostCmd.java:88) > > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156) > > at com.cloud.api.ApiServer.queueCommand(ApiServer.java:764) > > at com.cloud.api.ApiServer.handleRequest(ApiServer.java:588) > > at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:321) > > at com.cloud.api.ApiServlet$1.run(ApiServlet.java:134) > > at > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55) > > at > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102) > > at > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52) > > at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:131) > > at com.cloud.api.ApiServlet.doGet(ApiServlet.java:93) > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:645) > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:750) > > at > > org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1386) > > at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:755) > > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:547) > > at > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) > > at > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > > at > > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) > > at > > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) > > at > > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) > > at > > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300) > > at > > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) > > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485) > > at > > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) > > at > > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) > > at > > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215) > > at > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) > > at > > org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:767) > > at > > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) > > at > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > > at org.eclipse.jetty.server.Server.handle(Server.java:500) > > at > org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) > > at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547) > > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) > > at > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) > > at > org.eclipse.jetty.io > .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) > > at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) > > at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) > > at > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) > > at > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) > > at > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) > > at > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) > > at > > org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) > > at > > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) > > at > > org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) > > at java.base/java.lang.Thread.run(Thread.java:834) > > On Wed, Apr 14, 2021 at 6:58 PM Nicolas Vazquez < > nicolas.vazq...@shapeblue.com> wrote: > > > Hi Hean, > > > > What does your management server logs show after the listHosts and the > > deleteHost API calls? > > > > > > Regards, > > > > Nicolas Vazquez > > > > ________________________________ > > From: Hean Seng <heans...@gmail.com> > > Sent: Tuesday, April 13, 2021 4:47 PM > > To: users@cloudstack.apache.org <users@cloudstack.apache.org> > > Subject: Request failed with status code 530 > > > > Hi > > > > I added a host to a wrong cluster, deleted it and re-add to the correct > > cluster . After that found that duplicate host inside the cluster , and > > after delete it the duplicated record, it come with error > > "Request failed with status code 530" > > > > MYSQL record ; > > > > 16 | lax-06 | Removed | Disabled | NULL > > > > 17 | lax-06 | Up | Enabled | 2130706689 > > > > > > Duplicated Host deleted is id=16 , but now the error happens is 17. > > > > In API : > > > > cmk listHosts id=17 > > > > 🙈 Error: (HTTP 530, error code 9999) <nil> > > > > for 16 , is return empty record. > > > > API log show" > > > > 2021-04-13 19:41:41,395 INFO [a.c.c.a.ApiServer] > > (qtp1644231115-1244:ctx-eba7bffd ctx-95331a28) (logid:1e346c12) (userId=2 > > accountId=2 sessionId=node01qfrx0iypf3ts1q1kf43fexp7u174) 127.0.0.1 -- > GET > > > > > command=listHosts&id=17&response=json&sessionkey=X3J4ui0AxuwP2zbYqrTrZxHzYLg > > 530 null > > > > I tried to forced delete host 17. but seems error also; > > > > cmk deletehost id=17 forced=true > > > > 🙈 Error: (HTTP 530, error code 9999) <nil> > > > > > > Any one help to give me idea of what's going on, and how to solve > > this problem > > > > > > > > > > -- > > Regards, > > Hean Seng > > > > nicolas.vazq...@shapeblue.com > > www.shapeblue.com<http://www.shapeblue.com> > > 3 London Bridge Street, 3rd floor, News Building, London SE1 9SGUK > > @shapeblue > > > > > > > > > > -- > Regards, > Hean Seng > > nicolas.vazq...@shapeblue.com > www.shapeblue.com<http://www.shapeblue.com> > 3 London Bridge Street, 3rd floor, News Building, London SE1 9SGUK > @shapeblue > > > > -- Regards, Hean Seng nicolas.vazq...@shapeblue.com www.shapeblue.com 3 London Bridge Street, 3rd floor, News Building, London SE1 9SGUK @shapeblue