GitHub user dpthakar created a discussion: Cloudstack 4.22 SSVM fails to start
Hello,
**Problem:-**
SSVM Fails to start.
**Versions:-**
Cloudstack agent 4.22
Cloudstack Management 4.22
ubuntu 24.04
**Error:-**
```
==> /var/log/cloudstack/agent/agent.log <==
2025-11-11 16:07:12,873 INFO [kvm.storage.LibvirtStorageAdaptor]
(AgentRequest-Handler-1:[]) (logid:6786196e) Trying to fetch storage pool
3b4573e2-11b4-3b02-81b3-a7d8f8c97101 from libvirt
2025-11-11 16:07:12,942 WARN [kvm.resource.BridgeVifDriver]
(AgentRequest-Handler-1:[]) (logid:6786196e) Execution of process [41853] for
command [/usr/share/cloudstack-common/scripts/vm/network/vnet/modifyvlan.sh -v
18 -p bond1 -b brbond1-18 -o add ] failed.
2025-11-11 16:07:12,943 WARN [kvm.resource.BridgeVifDriver]
(AgentRequest-Handler-1:[]) (logid:6786196e) Process [41853] for command
[/usr/share/cloudstack-common/scripts/vm/network/vnet/modifyvlan.sh -v 18 -p
bond1 -b brbond1-18 -o add ] encountered the error: [Error: 8021q: VLAN device
already
exists./usr/share/cloudstack-common/scripts/vm/network/vnet/modifyvlan.sh: line
37: /proc/sys/net/ipv6/conf/bond1.18/disable_ipv6: No such file or
directoryCannot find device "bond1.18"Failed to create vlan 18 on pif: bond1.].
2025-11-11 16:07:12,943 WARN [resource.wrapper.LibvirtStartCommandWrapper]
(AgentRequest-Handler-1:[]) (logid:6786196e) InternalErrorException
com.cloud.exception.InternalErrorException: Failed to create vnet 18: Error:
8021q: VLAN device already
exists./usr/share/cloudstack-common/scripts/vm/network/vnet/modifyvlan.sh: line
37: /proc/sys/net/ipv6/conf/bond1.18/disable_ipv6: No such file or
directoryCannot find device "bond1.18"Failed to create vlan 18 on pif: bond1.
at
com.cloud.hypervisor.kvm.resource.BridgeVifDriver.createVnet(BridgeVifDriver.java:338)
at
com.cloud.hypervisor.kvm.resource.BridgeVifDriver.createVnetBr(BridgeVifDriver.java:320)
at
com.cloud.hypervisor.kvm.resource.BridgeVifDriver.createStorageVnetBridgeIfNeeded(BridgeVifDriver.java:203)
at
com.cloud.hypervisor.kvm.resource.BridgeVifDriver.plug(BridgeVifDriver.java:272)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.createVif(LibvirtComputingResource.java:3861)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.createVifs(LibvirtComputingResource.java:3438)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStartCommandWrapper.execute(LibvirtStartCommandWrapper.java:86)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStartCommandWrapper.execute(LibvirtStartCommandWrapper.java:52)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:2280)
at com.cloud.agent.Agent.processRequest(Agent.java:813)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1295)
at com.cloud.utils.nio.Task.call(Task.java:83)
at com.cloud.utils.nio.Task.call(Task.java:29)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
```
Earlier SSVM was working fine on CS Agent 4.20, after upgrading CS Agent to
4.22 AND CS Mgmt to 4,22 we are getting above mentioned error.
_Below is current network config._
```
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group
default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host noprefixroute
valid_lft forever preferred_lft forever
2: ens3f0np0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group
default qlen 1000
link/ether 7c:c2:55:42:8d:f4 brd ff:ff:ff:ff:ff:ff
altname enp99s0f0np0
3: ens3f1np1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group
default qlen 1000
link/ether 7c:c2:55:42:8d:f5 brd ff:ff:ff:ff:ff:ff
altname enp99s0f1np1
4: ens2f0np0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9214 qdisc mq master
bond0 state UP group default qlen 1000
link/ether 0a:b7:ba:ab:5b:13 brd ff:ff:ff:ff:ff:ff permaddr
7c:c2:55:42:8b:54
altname enp68s0f0np0
5: ens2f1np1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 9214 qdisc mq state DOWN
group default qlen 1000
link/ether 7c:c2:55:42:8b:55 brd ff:ff:ff:ff:ff:ff
altname enp68s0f1np1
6: enp3s0f0np0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group
default qlen 1000
link/ether 7c:c2:55:42:8e:34 brd ff:ff:ff:ff:ff:ff
7: enp3s0f1np1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group
default qlen 1000
link/ether 7c:c2:55:42:8e:35 brd ff:ff:ff:ff:ff:ff
8: enp36s0f0np0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9214 qdisc mq
master bond0 state UP group default qlen 1000
link/ether 0a:b7:ba:ab:5b:13 brd ff:ff:ff:ff:ff:ff permaddr
7c:c2:55:42:85:44
9: enp36s0f1np1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9214 qdisc mq
master bond1 state UP group default qlen 1000
link/ether ca:9f:8f:ec:8e:1f brd ff:ff:ff:ff:ff:ff permaddr
7c:c2:55:42:85:45
10: enp227s0f0np0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group
default qlen 1000
link/ether 7c:c2:55:42:85:24 brd ff:ff:ff:ff:ff:ff
11: enp51s0f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN
group default qlen 1000
link/ether 7c:c2:55:7b:58:d6 brd ff:ff:ff:ff:ff:ff
12: enp227s0f1np1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9214 qdisc mq
master bond1 state UP group default qlen 1000
link/ether ca:9f:8f:ec:8e:1f brd ff:ff:ff:ff:ff:ff permaddr
7c:c2:55:42:85:25
13: enp196s0f0np0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9214 qdisc mq
master bond0 state UP group default qlen 1000
link/ether 0a:b7:ba:ab:5b:13 brd ff:ff:ff:ff:ff:ff permaddr
7c:c2:55:42:84:84
14: enp196s0f1np1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 9214 qdisc mq state
DOWN group default qlen 1000
link/ether 7c:c2:55:42:84:85 brd ff:ff:ff:ff:ff:ff
15: enp131s0f0np0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group
default qlen 1000
link/ether 7c:c2:55:42:85:34 brd ff:ff:ff:ff:ff:ff
16: enp131s0f1np1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group
default qlen 1000
link/ether 7c:c2:55:42:85:35 brd ff:ff:ff:ff:ff:ff
17: enp164s0f0np0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9214 qdisc mq
master bond0 state UP group default qlen 1000
link/ether 0a:b7:ba:ab:5b:13 brd ff:ff:ff:ff:ff:ff permaddr
7c:c2:55:42:85:54
18: enp164s0f1np1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 9214 qdisc mq state
DOWN group default qlen 1000
link/ether 7c:c2:55:42:85:55 brd ff:ff:ff:ff:ff:ff
19: enp51s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master
hostbr0 state UP group default qlen 1000
link/ether 7c:c2:55:7b:58:d7 brd ff:ff:ff:ff:ff:ff
20: enxbe3af2b6059f: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group
default qlen 1000
link/ether be:3a:f2:b6:05:9f brd ff:ff:ff:ff:ff:ff
21: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 9214 qdisc noqueue
master cloudbr0 state UP group default qlen 1000
link/ether 0a:b7:ba:ab:5b:13 brd ff:ff:ff:ff:ff:ff
22: bond1: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 9214 qdisc noqueue
state UP group default qlen 1000
link/ether ca:9f:8f:ec:8e:1f brd ff:ff:ff:ff:ff:ff
inet6 fe80::c89f:8fff:feec:8e1f/64 scope link
valid_lft forever preferred_lft forever
23: cloudbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9214 qdisc noqueue state UP
group default qlen 1000
link/ether 76:55:ab:fd:45:64 brd ff:ff:ff:ff:ff:ff
inet6 fe80::7455:abff:fefd:4564/64 scope link
valid_lft forever preferred_lft forever
24: csmgmtbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9214 qdisc noqueue state
UP group default qlen 1000
link/ether da:30:cd:f0:20:0a brd ff:ff:ff:ff:ff:ff
inet 192.168.10.131/23 metric 100 brd 192.168.11.255 scope global dynamic
csmgmtbr0
valid_lft 7773596sec preferred_lft 7773596sec
inet6 fe80::d830:cdff:fef0:200a/64 scope link
valid_lft forever preferred_lft forever
25: hostbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
group default qlen 1000
link/ether fe:e0:4f:7c:cb:40 brd ff:ff:ff:ff:ff:ff
inet 192.168.14.116/23 metric 100 brd 192.168.15.255 scope global dynamic
hostbr0
valid_lft 7773572sec preferred_lft 7773572sec
inet6 fe80::fce0:4fff:fe7c:cb40/64 scope link
valid_lft forever preferred_lft forever
26: strbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9214 qdisc noqueue state UP
group default qlen 1000
link/ether 7e:44:09:e3:06:76 brd ff:ff:ff:ff:ff:ff
inet 192.168.18.105/24 brd 192.168.18.255 scope global strbr0
valid_lft forever preferred_lft forever
inet6 fe80::7c44:9ff:fee3:676/64 scope link
valid_lft forever preferred_lft forever
27: vlan18@bond1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9214 qdisc noqueue
master strbr0 state UP group default qlen 1000
link/ether ca:9f:8f:ec:8e:1f brd ff:ff:ff:ff:ff:ff
28: vlan10@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9214 qdisc noqueue
master csmgmtbr0 state UP group default qlen 1000
link/ether 0a:b7:ba:ab:5b:13 brd ff:ff:ff:ff:ff:ff
29: cloud0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
group default qlen 1000
link/ether 4a:3b:6e:d2:39:6d brd ff:ff:ff:ff:ff:ff
inet 169.254.0.1/16 scope global cloud0
valid_lft forever preferred_lft forever
inet6 fe80::483b:6eff:fed2:396d/64 scope link
valid_lft forever preferred_lft forever
42: bond0.4090@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9214 qdisc noqueue
master brbond0-4090 state UP group default qlen 1000
link/ether 0a:b7:ba:ab:5b:13 brd ff:ff:ff:ff:ff:ff
43: brbond0-4090: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9214 qdisc noqueue
state UP group default qlen 1000
link/ether c2:88:40:83:d0:27 brd ff:ff:ff:ff:ff:ff
44: vnet6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master
cloud0 state UNKNOWN group default qlen 1000
link/ether fe:00:a9:fe:3c:68 brd ff:ff:ff:ff:ff:ff
inet6 fe80::fc00:a9ff:fefe:3c68/64 scope link
valid_lft forever preferred_lft forever
45: vnet7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9214 qdisc noqueue master
csmgmtbr0 state UNKNOWN group default qlen 1000
link/ether fe:00:48:00:00:0b brd ff:ff:ff:ff:ff:ff
inet6 fe80::fc00:48ff:fe00:b/64 scope link
valid_lft forever preferred_lft forever
46: vnet8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9214 qdisc noqueue master
brbond0-4090 state UNKNOWN group default qlen 1000
link/ether fe:00:9f:00:00:0c brd ff:ff:ff:ff:ff:ff
inet6 fe80::fc00:9fff:fe00:c/64 scope link
valid_lft forever preferred_lft forever
```
To fix it, we tried following but did not help.
Reboot all hosts & mgmgt
Downgrade CS Agent to 4.20
Erase CS Agent package & re-add host
GitHub link: https://github.com/apache/cloudstack/discussions/12064
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
