kiranchavala opened a new issue, #14122:
URL: https://github.com/apache/cloudstack/issues/14122
### problem
Scaling issues with cks cluster
### versions
ACS 4.23, 4.22
Hypervisor : KVM
CKS : 1.36
### The steps to reproduce the bug
Steps to reproduce the issue
1. Create 2 service offerings
2. create kubernetescluster name=cks-npe
zoneid=157cb8ca-1c40-4a1f-a737-8bddb1dcb2fd
kubernetesversionid=e8d2b996-54c8-457a-9a54-985ebc46b332
networkid=f2d97bea-3e96-44bd-ae16-fd98d7ea8a8e controlnodes=1 size=2
nodeofferings[0].node=control
nodeofferings[0].offering=bf332776-e6c1-43cb-922d-7de6bb5d38a7
nodeofferings[1].node=worker
nodeofferings[1].offering=d7892a54-8604-44ab-8892-dcd1357571a0 hypervisor=KVM
Check the database
```
mysql> SELECT name, state, service_offering_id,
control_node_service_offering_id, worker_node_service_offering_id,
etcd_node_service_offering_id, etcd_node_count FROM cloud.kubernetes_cluster
WHERE name='cks-npe' AND removed IS NULL;
+---------+---------+---------------------+----------------------------------+---------------------------------+-------------------------------+-----------------+
| name | state | service_offering_id | control_node_service_offering_id
| worker_node_service_offering_id | etcd_node_service_offering_id |
etcd_node_count |
+---------+---------+---------------------+----------------------------------+---------------------------------+-------------------------------+-----------------+
| cks-npe | Running | NULL | 26
| 27 | NULL |
NULL |
+---------+---------+---------------------+----------------------------------+---------------------------------+-------------------------------+-----------------+
1 row in set (0.00 sec)
```
3. scale kubernetescluster id=7944a42f-fcf2-4f4e-932a-e7b59e0724b1 size=3
serviceofferingid=d7892a54-8604-44ab-8892-dcd1357571a0
exception
```
(localcloud) 🐱 > scale kubernetescluster
id=7944a42f-fcf2-4f4e-932a-e7b59e0724b1 size=3
serviceofferingid=d7892a54-8604-44ab-8892-dcd1357571a0
{
"account": "admin",
"accountid": "47dc9e4c-83f6-11f1-aed8-bc24112069f3",
"cmd":
"org.apache.cloudstack.api.command.user.kubernetes.cluster.ScaleKubernetesClusterCmd",
"completed": "2026-09-10T10:41:08+0000",
"created": "2026-09-10T10:41:08+0000",
"domainid": "c92b36a9-83f5-11f1-aed8-bc24112069f3",
"domainpath": "ROOT",
"jobid": "034171ef-25d7-4243-bbd7-6524431309af",
"jobprocstatus": 0,
"jobresult": {
"errorcode": 530,
"errortext": "Cannot invoke
\"com.cloud.offering.ServiceOffering.getRamSize()\" because
\"existingServiceOffering\" is null"
},
"jobresultcode": 530,
"jobresulttype": "object",
"jobstatus": 2,
"userid": "47dcda68-83f6-11f1-aed8-bc24112069f3"
}
```
Logs
```
root@Cloudstack-423:/home/ubuntu# cat
/var/log/cloudstack/management/management-server.log |grep -i "logid:034171ef"
2026-09-10 10:41:08,704 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl$5]
(API-Job-Executor-22:[ctx-606a2296, job-1430]) (logid:034171ef) Executing
AsyncJob
{"accountId":2,"cmd":"org.apache.cloudstack.api.command.user.kubernetes.cluster.ScaleKubernetesClusterCmd","cmdInfo":"{\"expires\":\"2026-09-10T10:56:08Z\",\"sessionkey\":\"v1xKdYPWziwzZfJE-1T4fxIgyyI\",\"httpmethod\":\"POST\",\"ctxAccountId\":\"2\",\"uuid\":\"7944a42f-fcf2-4f4e-932a-e7b59e0724b1\",\"cmdEventType\":\"KUBERNETES.CLUSTER.SCALE\",\"signatureversion\":\"3\",\"size\":\"3\",\"serviceofferingid\":\"d7892a54-8604-44ab-8892-dcd1357571a0\",\"response\":\"json\",\"ctxUserId\":\"2\",\"ctxStartEventId\":\"9114\",\"id\":\"7944a42f-fcf2-4f4e-932a-e7b59e0724b1\",\"ctxDetails\":\"{\\\"interface
com.cloud.kubernetes.cluster.KubernetesCluster\\\":\\\"7944a42f-fcf2-4f4e-932a-e7b59e0724b1\\\",\\\"interface
com.cloud.offering.ServiceOffering\\\":\\\"d7892a54-8604-44ab-8892-dcd1357571a0\\\"}\"}","cmdVersion":0,"completeMsid":null,"created":
null,"id":1430,"initMsid":206863092181491,"instanceId":null,"instanceType":"KubernetesCluster","lastPolled":null,"lastUpdated":null,"processStatus":0,"removed":null,"result":null,"resultCode":0,"status":"IN_PROGRESS","userId":2,"uuid":"034171ef-25d7-4243-bbd7-6524431309af"}
2026-09-10 10:41:08,706 DEBUG [c.c.u.AccountManagerImpl]
(API-Job-Executor-22:[ctx-606a2296, job-1430, ctx-e097f2dc]) (logid:034171ef)
Account [Account
[{"accountName":"admin","id":2,"uuid":"47dc9e4c-83f6-11f1-aed8-bc24112069f3"}]]
has access to resource.
2026-09-10 10:41:08,735 ERROR [c.c.a.ApiAsyncJobDispatcher]
(API-Job-Executor-22:[ctx-606a2296, job-1430]) (logid:034171ef) Unexpected
exception while executing
org.apache.cloudstack.api.command.user.kubernetes.cluster.ScaleKubernetesClusterCmd
java.lang.NullPointerException: Cannot invoke
"com.cloud.offering.ServiceOffering.getRamSize()" because
"existingServiceOffering" is null
2026-09-10 10:41:08,737 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-22:[ctx-606a2296, job-1430]) (logid:034171ef) Complete async
job-1430, jobStatus: FAILED, resultCode: 530, result:
org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Cannot
invoke "com.cloud.offering.ServiceOffering.getRamSize()" because
"existingServiceOffering" is null"}
2026-09-10 10:41:08,737 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-22:[ctx-606a2296, job-1430]) (logid:034171ef) Publish async
job-1430 complete on message bus
2026-09-10 10:41:08,737 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-22:[ctx-606a2296, job-1430]) (logid:034171ef) Wake up jobs
related to job-1430
2026-09-10 10:41:08,737 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-22:[ctx-606a2296, job-1430]) (logid:034171ef) Update db
status for job-1430
2026-09-10 10:41:08,738 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-22:[ctx-606a2296, job-1430]) (logid:034171ef) Wake up jobs
joined with job-1430 and disjoin all subjobs created from job- 1430
2026-09-10 10:41:08,751 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl$5]
(API-Job-Executor-22:[ctx-606a2296, job-1430]) (logid:034171ef) Done executing
org.apache.cloudstack.api.command.user.kubernetes.cluster.ScaleKubernetesClusterCmd
for job-1430
2026-09-10 10:41:08,751 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(API-Job-Executor-22:[ctx-606a2296, job-1430]) (logid:034171ef) Remove job-1430
from job monitoring
```
4. Scale the cks cluster by providing only the size parameter
```
(localcloud) 🐱 > scale kubernetescluster
id=7944a42f-fcf2-4f4e-932a-e7b59e0724b1 size=3
{
"account": "admin",
"accountid": "47dc9e4c-83f6-11f1-aed8-bc24112069f3",
"cmd":
"org.apache.cloudstack.api.command.user.kubernetes.cluster.ScaleKubernetesClusterCmd",
"completed": "2026-09-10T10:57:23+0000",
"created": "2026-09-10T10:57:23+0000",
"domainid": "c92b36a9-83f5-11f1-aed8-bc24112069f3",
"domainpath": "ROOT",
"jobid": "0f928699-0ce2-47de-b9db-eeb8401c406d",
"jobprocstatus": 0,
"jobresult": {
"errorcode": 530,
"errortext": "The Kubernetes cluster cks-npe does not have a service
offering set for node type ETCD"
},
"jobresultcode": 530,
"jobresulttype": "object",
"jobstatus": 2,
"userid": "47dcda68-83f6-11f1-aed8-bc24112069f3"
}
🙈 Error: async API failed for job 0f928699-0ce2-47de-b9db-eeb8401c406d
```
Logs
```
root@Cloudstack-423:/home/ubuntu# cat
/var/log/cloudstack/management/management-server.log |grep -i "logid:0f928699"
2026-09-10 10:57:23,386 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl$5]
(API-Job-Executor-23:[ctx-3a32d20f, job-1431]) (logid:0f928699) Executing
AsyncJob
{"accountId":2,"cmd":"org.apache.cloudstack.api.command.user.kubernetes.cluster.ScaleKubernetesClusterCmd","cmdInfo":"{\"expires\":\"2026-09-10T11:12:23Z\",\"sessionkey\":\"CaP7TfxTrU4jexSnhOGcatQMFt8\",\"httpmethod\":\"POST\",\"ctxAccountId\":\"2\",\"uuid\":\"7944a42f-fcf2-4f4e-932a-e7b59e0724b1\",\"cmdEventType\":\"KUBERNETES.CLUSTER.SCALE\",\"signatureversion\":\"3\",\"size\":\"3\",\"response\":\"json\",\"ctxUserId\":\"2\",\"ctxStartEventId\":\"9118\",\"id\":\"7944a42f-fcf2-4f4e-932a-e7b59e0724b1\",\"ctxDetails\":\"{\\\"interface
com.cloud.kubernetes.cluster.KubernetesCluster\\\":\\\"7944a42f-fcf2-4f4e-932a-e7b59e0724b1\\\"}\"}","cmdVersion":0,"completeMsid":null,"created":null,"id":1431,"initMsid":206863092181491,"instanceId":null,"instanceType":"KubernetesCluster","lastPolled":null,"lastUpdated":null,"processStatus":0,"removed":n
ull,"result":null,"resultCode":0,"status":"IN_PROGRESS","userId":2,"uuid":"0f928699-0ce2-47de-b9db-eeb8401c406d"}
2026-09-10 10:57:23,410 INFO [c.c.k.c.a.KubernetesClusterScaleWorker]
(API-Job-Executor-23:[ctx-3a32d20f, job-1431, ctx-eca8a9bb]) (logid:0f928699)
Scaling Kubernetes cluster KubernetesCluster
{"id":7,"name":"cks-npe","uuid":"7944a42f-fcf2-4f4e-932a-e7b59e0724b1"}
2026-09-10 10:57:23,412 ERROR [c.c.k.c.a.KubernetesClusterScaleWorker]
(API-Job-Executor-23:[ctx-3a32d20f, job-1431, ctx-eca8a9bb]) (logid:0f928699)
The Kubernetes cluster cks-npe does not have a service offering set for node
type ETCD
2026-09-10 10:57:23,412 INFO [c.c.k.c.a.KubernetesClusterScaleWorker]
(API-Job-Executor-23:[ctx-3a32d20f, job-1431, ctx-eca8a9bb]) (logid:0f928699)
Revoking launch permission for systemVM Template
2026-09-10 10:57:23,434 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-23:[ctx-3a32d20f, job-1431]) (logid:0f928699) Complete async
job-1431, jobStatus: FAILED, resultCode: 530, result:
org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"The
Kubernetes cluster cks-npe does not have a service offering set for node type
ETCD"}
2026-09-10 10:57:23,435 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-23:[ctx-3a32d20f, job-1431]) (logid:0f928699) Publish async
job-1431 complete on message bus
2026-09-10 10:57:23,435 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-23:[ctx-3a32d20f, job-1431]) (logid:0f928699) Wake up jobs
related to job-1431
2026-09-10 10:57:23,435 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-23:[ctx-3a32d20f, job-1431]) (logid:0f928699) Update db
status for job-1431
2026-09-10 10:57:23,435 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-23:[ctx-3a32d20f, job-1431]) (logid:0f928699) Wake up jobs
joined with job-1431 and disjoin all subjobs created from job- 1431
2026-09-10 10:57:23,445 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl$5]
(API-Job-Executor-23:[ctx-3a32d20f, job-1431]) (logid:0f928699) Done executing
org.apache.cloudstack.api.command.user.kubernetes.cluster.ScaleKubernetesClusterCmd
for job-1431
2026-09-10 10:57:23,445 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(API-Job-Executor-23:[ctx-3a32d20f, job-1431]) (logid:0f928699) Remove job-1431
from job monitoring
```
5. scale kubernetescluster id=7944a42f-fcf2-4f4e-932a-e7b59e0724b1 size=3
nodeofferings[0].node=etcd
nodeofferings[0].offering=d7892a54-8604-44ab-8892-dcd1357571a0
```
(localcloud) 🐱 > scale kubernetescluster
id=7944a42f-fcf2-4f4e-932a-e7b59e0724b1 size=3 nodeofferings[0].node=etcd
nodeofferings[0].offering=d7892a54-8604-44ab-8892-dcd1357571a0
{
"account": "admin",
"accountid": "47dc9e4c-83f6-11f1-aed8-bc24112069f3",
"cmd":
"org.apache.cloudstack.api.command.user.kubernetes.cluster.ScaleKubernetesClusterCmd",
"completed": "2026-09-10T11:04:52+0000",
"created": "2026-09-10T11:04:51+0000",
"domainid": "c92b36a9-83f5-11f1-aed8-bc24112069f3",
"domainpath": "ROOT",
"jobid": "8dabeecf-0c6a-4771-b2d9-502865ffd944",
"jobprocstatus": 0,
"jobresult": {
"errorcode": 530,
"errortext": "Cannot invoke
\"com.cloud.offering.ServiceOffering.getRamSize()\" because
\"existingServiceOffering\" is null"
},
"jobresultcode": 530,
"jobresulttype": "object",
"jobstatus": 2,
"userid": "47dcda68-83f6-11f1-aed8-bc24112069f3"
}
🙈 Error: async API failed for job 8dabeecf-0c6a-4771-b2d9-502865ffd944
```
logs
```
root@Cloudstack-423:/home/ubuntu# cat
/var/log/cloudstack/management/management-server.log |grep -i "logid:8dabeecf"
2026-09-10 11:04:51,999 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl$5]
(API-Job-Executor-24:[ctx-ed8d2ed5, job-1432]) (logid:8dabeecf) Executing
AsyncJob
{"accountId":2,"cmd":"org.apache.cloudstack.api.command.user.kubernetes.cluster.ScaleKubernetesClusterCmd","cmdInfo":"{\"expires\":\"2026-09-10T11:19:51Z\",\"sessionkey\":\"CaP7TfxTrU4jexSnhOGcatQMFt8\",\"httpmethod\":\"POST\",\"nodeofferings[0].offering\":\"d7892a54-8604-44ab-8892-dcd1357571a0\",\"ctxAccountId\":\"2\",\"uuid\":\"7944a42f-fcf2-4f4e-932a-e7b59e0724b1\",\"nodeofferings[0].node\":\"etcd\",\"cmdEventType\":\"KUBERNETES.CLUSTER.SCALE\",\"signatureversion\":\"3\",\"size\":\"3\",\"response\":\"json\",\"ctxUserId\":\"2\",\"ctxStartEventId\":\"9121\",\"id\":\"7944a42f-fcf2-4f4e-932a-e7b59e0724b1\",\"ctxDetails\":\"{\\\"interface
com.cloud.kubernetes.cluster.KubernetesCluster\\\":\\\"7944a42f-fcf2-4f4e-932a-e7b59e0724b1\\\"}\"}","cmdVersion":0,"completeMsid":null,"created":null,"id":1432,"initMsid":206863092181491,"instanceId":
null,"instanceType":"KubernetesCluster","lastPolled":null,"lastUpdated":null,"processStatus":0,"removed":null,"result":null,"resultCode":0,"status":"IN_PROGRESS","userId":2,"uuid":"8dabeecf-0c6a-4771-b2d9-502865ffd944"}
2026-09-10 11:04:52,015 DEBUG [c.c.k.c.KubernetesServiceHelperImpl]
(API-Job-Executor-24:[ctx-ed8d2ed5, job-1432, ctx-75ed24fb]) (logid:8dabeecf)
Node Type: 'etcd' should use Service Offering ID:
'd7892a54-8604-44ab-8892-dcd1357571a0'
2026-09-10 11:04:52,027 ERROR [c.c.a.ApiAsyncJobDispatcher]
(API-Job-Executor-24:[ctx-ed8d2ed5, job-1432]) (logid:8dabeecf) Unexpected
exception while executing
org.apache.cloudstack.api.command.user.kubernetes.cluster.ScaleKubernetesClusterCmd
java.lang.NullPointerException: Cannot invoke
"com.cloud.offering.ServiceOffering.getRamSize()" because
"existingServiceOffering" is null
2026-09-10 11:04:52,028 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-24:[ctx-ed8d2ed5, job-1432]) (logid:8dabeecf) Complete async
job-1432, jobStatus: FAILED, resultCode: 530, result:
org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Cannot
invoke "com.cloud.offering.ServiceOffering.getRamSize()" because
"existingServiceOffering" is null"}
2026-09-10 11:04:52,028 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-24:[ctx-ed8d2ed5, job-1432]) (logid:8dabeecf) Publish async
job-1432 complete on message bus
2026-09-10 11:04:52,028 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-24:[ctx-ed8d2ed5, job-1432]) (logid:8dabeecf) Wake up jobs
related to job-1432
2026-09-10 11:04:52,028 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-24:[ctx-ed8d2ed5, job-1432]) (logid:8dabeecf) Update db
status for job-1432
2026-09-10 11:04:52,029 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-24:[ctx-ed8d2ed5, job-1432]) (logid:8dabeecf) Wake up jobs
joined with job-1432 and disjoin all subjobs created from job- 1432
2026-09-10 11:04:52,044 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl$5]
(API-Job-Executor-24:[ctx-ed8d2ed5, job-1432]) (logid:8dabeecf) Done executing
org.apache.cloudstack.api.command.user.kubernetes.cluster.ScaleKubernetesClusterCmd
for job-1432
2026-09-10 11:04:52,044 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(API-Job-Executor-24:[ctx-ed8d2ed5, job-1432]) (logid:8dabeecf) Remove job-1432
from job monitoring
```
6. scale kubernetescluster id=7944a42f-fcf2-4f4e-932a-e7b59e0724b1
nodeofferings[0].key=etcd
nodeofferings[0].offering=d7892a54-8604-44ab-8892-dcd1357571a0
```
(localcloud) 🐱 > scale kubernetescluster
id=7944a42f-fcf2-4f4e-932a-e7b59e0724b1 nodeofferings[0].key=etcd
nodeofferings[0].offering=d7892a54-8604-44ab-8892-dcd1357571a0
{
"account": "admin",
"accountid": "47dc9e4c-83f6-11f1-aed8-bc24112069f3",
"cmd":
"org.apache.cloudstack.api.command.user.kubernetes.cluster.ScaleKubernetesClusterCmd",
"completed": "2026-09-10T11:07:43+0000",
"created": "2026-09-10T11:07:43+0000",
"domainid": "c92b36a9-83f5-11f1-aed8-bc24112069f3",
"domainpath": "ROOT",
"jobid": "051ee202-e648-438e-bd19-f1aa885044f4",
"jobprocstatus": 0,
"jobresult": {
"errorcode": 530,
"errortext": "Incomplete Node Type to Service Offering ID mapping:
'null' -> 'd7892a54-8604-44ab-8892-dcd1357571a0'"
},
"jobresultcode": 530,
"jobresulttype": "object",
"jobstatus": 2,
"userid": "47dcda68-83f6-11f1-aed8-bc24112069f3"
}
🙈 Error: async API failed for job 051ee202-e648-438e-bd19-f1aa885044f4
```
logs
```
root@Cloudstack-423:/home/ubuntu# cat
/var/log/cloudstack/management/management-server.log |grep -i "logid:051ee202"
2026-09-10 11:07:43,553 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl$5]
(API-Job-Executor-25:[ctx-0ec6aa90, job-1433]) (logid:051ee202) Executing
AsyncJob
{"accountId":2,"cmd":"org.apache.cloudstack.api.command.user.kubernetes.cluster.ScaleKubernetesClusterCmd","cmdInfo":"{\"expires\":\"2026-09-10T11:22:43Z\",\"sessionkey\":\"CaP7TfxTrU4jexSnhOGcatQMFt8\",\"httpmethod\":\"POST\",\"nodeofferings[0].key\":\"etcd\",\"nodeofferings[0].offering\":\"d7892a54-8604-44ab-8892-dcd1357571a0\",\"ctxAccountId\":\"2\",\"uuid\":\"7944a42f-fcf2-4f4e-932a-e7b59e0724b1\",\"cmdEventType\":\"KUBERNETES.CLUSTER.SCALE\",\"signatureversion\":\"3\",\"response\":\"json\",\"ctxUserId\":\"2\",\"ctxStartEventId\":\"9124\",\"id\":\"7944a42f-fcf2-4f4e-932a-e7b59e0724b1\",\"ctxDetails\":\"{\\\"interface
com.cloud.kubernetes.cluster.KubernetesCluster\\\":\\\"7944a42f-fcf2-4f4e-932a-e7b59e0724b1\\\"}\"}","cmdVersion":0,"completeMsid":null,"created":null,"id":1433,"initMsid":206863092181491,"instanceId":null,"instanceTy
pe":"KubernetesCluster","lastPolled":null,"lastUpdated":null,"processStatus":0,"removed":null,"result":null,"resultCode":0,"status":"IN_PROGRESS","userId":2,"uuid":"051ee202-e648-438e-bd19-f1aa885044f4"}
2026-09-10 11:07:43,564 ERROR [c.c.k.c.KubernetesServiceHelperImpl]
(API-Job-Executor-25:[ctx-0ec6aa90, job-1433, ctx-6bdd9a94]) (logid:051ee202)
Incomplete Node Type to Service Offering ID mapping: 'null' ->
'd7892a54-8604-44ab-8892-dcd1357571a0'
2026-09-10 11:07:43,575 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-25:[ctx-0ec6aa90, job-1433]) (logid:051ee202) Complete async
job-1433, jobStatus: FAILED, resultCode: 530, result:
org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Incomplete
Node Type to Service Offering ID mapping: 'null' ->
'd7892a54-8604-44ab-8892-dcd1357571a0'"}
2026-09-10 11:07:43,575 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-25:[ctx-0ec6aa90, job-1433]) (logid:051ee202) Publish async
job-1433 complete on message bus
2026-09-10 11:07:43,575 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-25:[ctx-0ec6aa90, job-1433]) (logid:051ee202) Wake up jobs
related to job-1433
2026-09-10 11:07:43,575 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-25:[ctx-0ec6aa90, job-1433]) (logid:051ee202) Update db
status for job-1433
2026-09-10 11:07:43,576 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-25:[ctx-0ec6aa90, job-1433]) (logid:051ee202) Wake up jobs
joined with job-1433 and disjoin all subjobs created from job- 1433
2026-09-10 11:07:43,585 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl$5]
(API-Job-Executor-25:[ctx-0ec6aa90, job-1433]) (logid:051ee202) Done executing
org.apache.cloudstack.api.command.user.kubernetes.cluster.ScaleKubernetesClusterCmd
for job-1433
2026-09-10 11:07:43,585 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(API-Job-Executor-25:[ctx-0ec6aa90, job-1433]) (logid:051ee202) Remove job-1433
from job monitoring
```
7. scale kubernetescluster id=7944a42f-fcf2-4f4e-932a-e7b59e0724b1
autoscalingenabled=true minsize=3 maxsize=5
serviceofferingid=d7892a54-8604-44ab-8892-dcd1357571a0
```
(localcloud) 🐱 > scale kubernetescluster
id=7944a42f-fcf2-4f4e-932a-e7b59e0724b1 autoscalingenabled=true minsize=3
maxsize=5 serviceofferingid=d7892a54-8604-44ab-8892-dcd1357571a0
{
"account": "admin",
"accountid": "47dc9e4c-83f6-11f1-aed8-bc24112069f3",
"cmd":
"org.apache.cloudstack.api.command.user.kubernetes.cluster.ScaleKubernetesClusterCmd",
"completed": "2026-09-10T11:12:07+0000",
"created": "2026-09-10T11:12:07+0000",
"domainid": "c92b36a9-83f5-11f1-aed8-bc24112069f3",
"domainpath": "ROOT",
"jobid": "1f5ab1ce-0cd6-43ff-b0c7-9e87c19507c3",
"jobprocstatus": 0,
"jobresult": {
"errorcode": 530,
"errortext": "Cannot invoke
\"com.cloud.offering.ServiceOffering.getRamSize()\" because
\"existingServiceOffering\" is null"
},
"jobresultcode": 530,
"jobresulttype": "object",
"jobstatus": 2,
"userid": "47dcda68-83f6-11f1-aed8-bc24112069f3"
}
🙈 Error: async API failed for job 1f5ab1ce-0cd6-43ff-b0c7-9e87c19507c3
```
```
root@Cloudstack-423:/home/ubuntu# cat
/var/log/cloudstack/management/management-server.log |grep -i "logid:1f5ab1ce"
2026-09-10 11:12:07,190 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl$5]
(API-Job-Executor-26:[ctx-39b4db84, job-1434]) (logid:1f5ab1ce) Executing
AsyncJob
{"accountId":2,"cmd":"org.apache.cloudstack.api.command.user.kubernetes.cluster.ScaleKubernetesClusterCmd","cmdInfo":"{\"expires\":\"2026-09-10T11:27:07Z\",\"autoscalingenabled\":\"true\",\"sessionkey\":\"CaP7TfxTrU4jexSnhOGcatQMFt8\",\"httpmethod\":\"POST\",\"minsize\":\"3\",\"ctxAccountId\":\"2\",\"uuid\":\"7944a42f-fcf2-4f4e-932a-e7b59e0724b1\",\"cmdEventType\":\"KUBERNETES.CLUSTER.SCALE\",\"signatureversion\":\"3\",\"serviceofferingid\":\"d7892a54-8604-44ab-8892-dcd1357571a0\",\"response\":\"json\",\"ctxUserId\":\"2\",\"ctxStartEventId\":\"9127\",\"id\":\"7944a42f-fcf2-4f4e-932a-e7b59e0724b1\",\"maxsize\":\"5\",\"ctxDetails\":\"{\\\"interface
com.cloud.kubernetes.cluster.KubernetesCluster\\\":\\\"7944a42f-fcf2-4f4e-932a-e7b59e0724b1\\\",\\\"interface
com.cloud.offering.ServiceOffering\\\":\\\"d7892a54-8604-44ab-8892-dcd1357571a0\\
\"}\"}","cmdVersion":0,"completeMsid":null,"created":null,"id":1434,"initMsid":206863092181491,"instanceId":null,"instanceType":"KubernetesCluster","lastPolled":null,"lastUpdated":null,"processStatus":0,"removed":null,"result":null,"resultCode":0,"status":"IN_PROGRESS","userId":2,"uuid":"1f5ab1ce-0cd6-43ff-b0c7-9e87c19507c3"}
2026-09-10 11:12:07,197 DEBUG [c.c.u.AccountManagerImpl]
(API-Job-Executor-26:[ctx-39b4db84, job-1434, ctx-17662367]) (logid:1f5ab1ce)
Account [Account
[{"accountName":"admin","id":2,"uuid":"47dc9e4c-83f6-11f1-aed8-bc24112069f3"}]]
has access to resource.
2026-09-10 11:12:07,225 ERROR [c.c.a.ApiAsyncJobDispatcher]
(API-Job-Executor-26:[ctx-39b4db84, job-1434]) (logid:1f5ab1ce) Unexpected
exception while executing
org.apache.cloudstack.api.command.user.kubernetes.cluster.ScaleKubernetesClusterCmd
java.lang.NullPointerException: Cannot invoke
"com.cloud.offering.ServiceOffering.getRamSize()" because
"existingServiceOffering" is null
2026-09-10 11:12:07,232 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-26:[ctx-39b4db84, job-1434]) (logid:1f5ab1ce) Complete async
job-1434, jobStatus: FAILED, resultCode: 530, result:
org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Cannot
invoke "com.cloud.offering.ServiceOffering.getRamSize()" because
"existingServiceOffering" is null"}
2026-09-10 11:12:07,233 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-26:[ctx-39b4db84, job-1434]) (logid:1f5ab1ce) Publish async
job-1434 complete on message bus
2026-09-10 11:12:07,233 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-26:[ctx-39b4db84, job-1434]) (logid:1f5ab1ce) Wake up jobs
related to job-1434
2026-09-10 11:12:07,233 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-26:[ctx-39b4db84, job-1434]) (logid:1f5ab1ce) Update db
status for job-1434
2026-09-10 11:12:07,233 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-26:[ctx-39b4db84, job-1434]) (logid:1f5ab1ce) Wake up jobs
joined with job-1434 and disjoin all subjobs created from job- 1434
2026-09-10 11:12:07,250 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl$5]
(API-Job-Executor-26:[ctx-39b4db84, job-1434]) (logid:1f5ab1ce) Done executing
org.apache.cloudstack.api.command.user.kubernetes.cluster.ScaleKubernetesClusterCmd
for job-1434
2026-09-10 11:12:07,250 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(API-Job-Executor-26:[ctx-39b4db84, job-1434]) (logid:1f5ab1ce) Remove job-1434
from job monitoring
```
8. Create a cks cluster with etcd node
create kubernetescluster name=cks-etcd3 \
zoneid=157cb8ca-1c40-4a1f-a737-8bddb1dcb2fd \
kubernetesversionid=e8d2b996-54c8-457a-9a54-985ebc46b332 \
networkid=f2d97bea-3e96-44bd-ae16-fd98d7ea8a8e \
controlnodes=1 size=2 etcdnodes=1 \
nodeofferings[0].node=control
nodeofferings[0].offering=bf332776-e6c1-43cb-922d-7de6bb5d38a7 \
nodeofferings[1].node=worker
nodeofferings[1].offering=d7892a54-8604-44ab-8892-dcd1357571a0 \
nodeofferings[2].node=etcd
nodeofferings[2].offering=bf332776-e6c1-43cb-922d-7de6bb5d38a7 \
hypervisor=KVM
mysql> SELECT name, state, service_offering_id,
control_node_service_offering_id, worker_node_service_offering_id,
etcd_node_service_offering_id, etcd_node_count FROM cloud.kubernetes_cluster
WHERE name='cks-etcd3' AND removed IS NULL;
+-----------+---------+---------------------+----------------------------------+---------------------------------+-------------------------------+-----------------+
| name | state | service_offering_id |
control_node_service_offering_id | worker_node_service_offering_id |
etcd_node_service_offering_id | etcd_node_count |
+-----------+---------+---------------------+----------------------------------+---------------------------------+-------------------------------+-----------------+
| cks-etcd3 | Running | NULL |
26 | 27 | 26 |
1 |
+-----------+---------+---------------------+----------------------------------+---------------------------------+-------------------------------+-----------------+
1 row in set (0.00 sec)
```
(localcloud) 🐱 > scale kubernetescluster
id=bfdaa852-cdfc-42f1-a7bb-1c50ff9a59da size=3
{
"account": "admin",
"accountid": "47dc9e4c-83f6-11f1-aed8-bc24112069f3",
"cmd":
"org.apache.cloudstack.api.command.user.kubernetes.cluster.ScaleKubernetesClusterCmd",
"completed": "2026-09-10T11:45:41+0000",
"created": "2026-09-10T11:45:41+0000",
"domainid": "c92b36a9-83f5-11f1-aed8-bc24112069f3",
"domainpath": "ROOT",
"jobid": "3ac3beef-852d-46a7-be6b-04c956006e9d",
"jobprocstatus": 0,
"jobresult": {
"errorcode": 530,
"errortext": "Scaling failed for Kubernetes cluster : cks-etcd3, cluster
service offering not found"
},
"jobresultcode": 530,
"jobresulttype": "object",
"jobstatus": 2,
"userid": "47dcda68-83f6-11f1-aed8-bc24112069f3"
}
🙈 Error: async API failed for job 3ac3beef-852d-46a7-be6b-04c956006e9d
```
```
root@Cloudstack-423:/home/ubuntu# cat
/var/log/cloudstack/management/management-server.log |grep -i "logid:3ac3beef"
2026-09-10 11:45:41,445 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl$5]
(API-Job-Executor-31:[ctx-f1091fa2, job-1464]) (logid:3ac3beef) Executing
AsyncJob
{"accountId":2,"cmd":"org.apache.cloudstack.api.command.user.kubernetes.cluster.ScaleKubernetesClusterCmd","cmdInfo":"{\"expires\":\"2026-09-10T12:00:41Z\",\"sessionkey\":\"-6T60SNJxljbmS-Daml5a3Wkwmo\",\"httpmethod\":\"POST\",\"ctxAccountId\":\"2\",\"uuid\":\"bfdaa852-cdfc-42f1-a7bb-1c50ff9a59da\",\"cmdEventType\":\"KUBERNETES.CLUSTER.SCALE\",\"signatureversion\":\"3\",\"size\":\"3\",\"response\":\"json\",\"ctxUserId\":\"2\",\"ctxStartEventId\":\"9271\",\"id\":\"bfdaa852-cdfc-42f1-a7bb-1c50ff9a59da\",\"ctxDetails\":\"{\\\"interface
com.cloud.kubernetes.cluster.KubernetesCluster\\\":\\\"bfdaa852-cdfc-42f1-a7bb-1c50ff9a59da\\\"}\"}","cmdVersion":0,"completeMsid":null,"created":null,"id":1464,"initMsid":206863092181491,"instanceId":null,"instanceType":"KubernetesCluster","lastPolled":null,"lastUpdated":null,"processStatus":0,"removed":n
ull,"result":null,"resultCode":0,"status":"IN_PROGRESS","userId":2,"uuid":"3ac3beef-852d-46a7-be6b-04c956006e9d"}
2026-09-10 11:45:41,465 INFO [c.c.k.c.a.KubernetesClusterScaleWorker]
(API-Job-Executor-31:[ctx-f1091fa2, job-1464, ctx-39991b87]) (logid:3ac3beef)
Scaling Kubernetes cluster KubernetesCluster
{"id":9,"name":"cks-etcd3","uuid":"bfdaa852-cdfc-42f1-a7bb-1c50ff9a59da"}
2026-09-10 11:45:41,469 WARN [c.c.k.c.a.KubernetesClusterScaleWorker]
(API-Job-Executor-31:[ctx-f1091fa2, job-1464, ctx-39991b87]) (logid:3ac3beef)
Scaling failed for Kubernetes cluster : cks-etcd3, cluster service offering not
found
2026-09-10 11:45:41,469 INFO [c.c.k.c.a.KubernetesClusterScaleWorker]
(API-Job-Executor-31:[ctx-f1091fa2, job-1464, ctx-39991b87]) (logid:3ac3beef)
Revoking launch permission for systemVM Template
2026-09-10 11:45:41,497 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-31:[ctx-f1091fa2, job-1464]) (logid:3ac3beef) Complete async
job-1464, jobStatus: FAILED, resultCode: 530, result:
org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Scaling
failed for Kubernetes cluster : cks-etcd3, cluster service offering not found"}
2026-09-10 11:45:41,497 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-31:[ctx-f1091fa2, job-1464]) (logid:3ac3beef) Publish async
job-1464 complete on message bus
2026-09-10 11:45:41,497 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-31:[ctx-f1091fa2, job-1464]) (logid:3ac3beef) Wake up jobs
related to job-1464
2026-09-10 11:45:41,497 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-31:[ctx-f1091fa2, job-1464]) (logid:3ac3beef) Update db
status for job-1464
2026-09-10 11:45:41,498 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-31:[ctx-f1091fa2, job-1464]) (logid:3ac3beef) Wake up jobs
joined with job-1464 and disjoin all subjobs created from job- 1464
2026-09-10 11:45:41,507 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl$5]
(API-Job-Executor-31:[ctx-f1091fa2, job-1464]) (logid:3ac3beef) Done executing
org.apache.cloudstack.api.command.user.kubernetes.cluster.ScaleKubernetesClusterCmd
for job-1464
2026-09-10 11:45:41,507 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(API-Job-Executor-31:[ctx-f1091fa2, job-1464]) (logid:3ac3beef) Remove job-1464
from job monitoring
```
### What to do about it?
Scaling should work without any exception message
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
