tuanhoangth1603 opened a new issue, #7503:
URL: https://github.com/apache/cloudstack/issues/7503
Hello,
Have a good day team !
I created a Kubernetes cluster with two control nodes and one worker. Then I
tried to manually scale up by adding three more workers, but an error message
appeared on the dashboard saying "Scaling failed for Kubernetes cluster:
K8S-Dev, unable to update network rules." I checked the management log and
found details about the error.
```
2023-05-09 18:13:38,304 DEBUG [c.c.u.d.T.Transaction]
(API-Job-Executor-105:ctx-567d2785 job-2167 ctx-c4f4e780) (logid:2277cf45)
Rolling back the transaction: Time = 29 Name = API-Job-Executor-105; called by
-TransactionLegacy.rollback:888-TransactionLegacy.removeUpTo:831-TransactionLegacy.close:655-Transaction.execute:38-FirewallManagerImpl.createFirewallRule:242-FirewallManagerImpl.createIngressFirewallRule:194-GeneratedMethodAccessor846.invoke:-1-DelegatingMethodAccessorImpl.invoke:43-Method.invoke:566-AopUtils.invokeJoinpointUsingReflection:344-ReflectiveMethodInvocation.invokeJoinpoint:198-ReflectiveMethodInvocation.proceed:163
2023-05-09 18:13:38,321 ERROR [c.c.k.c.a.KubernetesClusterActionWorker]
(API-Job-Executor-105:ctx-567d2785 job-2167 ctx-c4f4e780) (logid:2277cf45)
Scaling failed for Kubernetes cluster : K8S-Dev, unable to update network rules
com.cloud.exception.NetworkRuleConflictException: The range specified,
2222-2227, conflicts with rule 216 which has 2222-2224
at
com.cloud.network.firewall.FirewallManagerImpl.detectRulesConflict(FirewallManagerImpl.java:470)
at
com.cloud.network.firewall.FirewallManagerImpl$1.doInTransaction(FirewallManagerImpl.java:255)
at
com.cloud.network.firewall.FirewallManagerImpl$1.doInTransaction(FirewallManagerImpl.java:242)
at com.cloud.utils.db.Transaction.execute(Transaction.java:40)
at
com.cloud.network.firewall.FirewallManagerImpl.createFirewallRule(FirewallManagerImpl.java:242)
at
com.cloud.network.firewall.FirewallManagerImpl.createIngressFirewallRule(FirewallManagerImpl.java:194)
at jdk.internal.reflect.GeneratedMethodAccessor846.invoke(Unknown
Source)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at
org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at
com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:52)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
at com.sun.proxy.$Proxy276.createIngressFirewallRule(Unknown Source)
at
com.cloud.kubernetes.cluster.actionworkers.KubernetesClusterResourceModifierActionWorker.provisionFirewallRules(KubernetesClusterResourceModifierActionWorker.java:446)
at
com.cloud.kubernetes.cluster.actionworkers.KubernetesClusterScaleWorker.scaleKubernetesClusterNetworkRules(KubernetesClusterScaleWorker.java:148)
at
com.cloud.kubernetes.cluster.actionworkers.KubernetesClusterScaleWorker.scaleUpKubernetesClusterSize(KubernetesClusterScaleWorker.java:369)
at
com.cloud.kubernetes.cluster.actionworkers.KubernetesClusterScaleWorker.scaleKubernetesClusterSize(KubernetesClusterScaleWorker.java:405)
at
com.cloud.kubernetes.cluster.actionworkers.KubernetesClusterScaleWorker.scaleCluster(KubernetesClusterScaleWorker.java:439)
at
com.cloud.kubernetes.cluster.KubernetesClusterManagerImpl.scaleKubernetesCluster(KubernetesClusterManagerImpl.java:1315)
at
org.apache.cloudstack.api.command.user.kubernetes.cluster.ScaleKubernetesClusterCmd.execute(ScaleKubernetesClusterCmd.java:162)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:163)
at
com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:106)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2023-05-09 18:13:38,333 INFO [c.c.k.c.a.KubernetesClusterActionWorker]
(API-Job-Executor-105:ctx-567d2785 job-2167 ctx-c4f4e780) (logid:2277cf45)
Revoking launch permission for systemVM template
2023-05-09 18:13:38,343 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-105:ctx-567d2785 job-2167) (logid:2277cf45) Complete async
job-2167, jobStatus: FAILED, resultCode: 530, result:
org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Scaling
failed for Kubernetes cluster : CFOX-Dev, unable to update network rules"}
```
I see the message "The range specified, 2222-2227, conflicts with rule 216
which has 2222-2224". Does CloudStack have a function to replace conflicting
rules? If not, how can we manually or auto-scale? My cluster has changed to
Alert status, and I cannot do anything now except stop or delete the cluster.
Additionally, how can I put my cluster back to Health status? I have checked,
and all pods in the cluster are still working fine.
Thanks !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
