I created this one with the pictures, https://github.com/apache/cloudstack/issues/10659
Pearl opened somethis similar: https://github.com/apache/cloudstack/issues/9920 Which seems related to one Pearl was working, but if this is for 4.21 is unrealistic from the time line perspective for me !!! and Wido have this working Something is really messed up ! Additionally: Issue 4) The VNI's I Choose for Guest are showing as VLANs, Wido what did you put on guest Vlan Range and Which Format Or nothing at all. Wido can you share which version of cloudstack are you running? Tata Y. > On Apr 4, 2025, at 6:27 AM, Chi vediamo <tatay...@gmail.com> wrote: > > > The cloud0 was added by the system, i did not created that. > I do not have a cloudbr0 > > what does it show for your in your agent.properties: public.network.device > and guest.network.properties > > > My agent.properties > private.network.device=cloudbr1 > guest.network.device=cloudbr1 > hypervisor.type=kvm > public.network.device=cloudbr1 > host=10.1.1.1@static > guid=579fafc9-5706-31f1-ae98-9f041ac90c80 > workers=5 > > the vxlan1000 I configured in the UI for the PUblic network and the > brvx-1000 that was added by Cloudstack, I did not do it manually >>> vxlan1000: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9050 qdisc noqueue master >>> brvx-1000 state UNKNOWN group default qlen 1000 >>> link/ether 1a:00:21:de:dc:bc brd ff:ff:ff:ff:ff:ff >>> brvx-1000: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9100 qdisc noqueue state >>> UP group default qlen 1000 >>> link/ether 1a:00:21:de:dc:bc brd ff:ff:ff:ff:ff:ff > > > I created a vxlan100 for the management and the management IPs are being > propagates > > my VNi > ~# vtysh -c 'sh evpn vni' > VNI Type VxLAN IF # MACs # ARPs # Remote VTEPs > Tenant VRF > 1000 L2 vxlan1000 2 0 0 > default <<< Added by Cloudstack Public but no Public IPs advertised > > 100 L2 vxlan100 7 11 4 > default <<< created manually for the initial clourbr1 and management sec > storage > 222 L2 vxlan222 10 20 9 > default <<< Separate vxlan for primary storage, its working > > My concerns are: Is normal the UI to show agent state gray for the 2 initial > SYSTEM-VMs the proxy and the secndarystorage >>>> ***Issue 3) s-VM and Proxy come up but Public IP addresses are not being >>>> routed Advertised.*** >>>> v-154-VM <public-IP-3> <Private-IP-2> <control-ip-169.x.x.x> >>>> [State-Running] [Agent State-Gray] >>>> s-285-VM <public-IP-2> <Private-IP-3> <control-ip-169.x.x.y> >>>> [State-Running] [Agent State-Gray] > > And where are my public IPs that should be added to the vxlan1000 ? > > > Also the UI does not allow me to add Public IPv6 address not even using > square brackets [] > > I can send snapshots but apache.org removes them. > > I know it should be a little detail but I do not know what I am missing, or > what is the normal status for the UI buttons. > > > Can any of you send me a snapshot of the UI configuration for the ZONe > > and for your zone for example Attached a PDF with the snapshots of mine. > the network storage I did not add any, cloudstack added itself to the cloudbr1 > > Also How do I Assign Public IPv6 Addresses ? > > thank you > > Tata Y. > > see PDF attached > > <TataY-Cloudstack-VXLAN-UI-2025-V1.pdf> > > > >> On Apr 4, 2025, at 5:00 AM, Wido den Hollander <w...@widodh.nl> wrote: >> >> Let me post a couple of outputs of a hypervisor we run: >> >> >> 9: vxlan100: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master >> cloudbr1 state UNKNOWN group default qlen 1000 >> link/ether 8e:7d:67:a7:c9:72 brd ff:ff:ff:ff:ff:ff >> inet6 fe80::8c7d:67ff:fea7:c972/64 scope link >> valid_lft forever preferred_lft forever >> 10: cloudbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state >> UP group default qlen 1000 >> link/ether 1e:11:99:1e:6d:46 brd ff:ff:ff:ff:ff:ff >> inet 10.100.2.2/20 scope global cloudbr1 >> valid_lft forever preferred_lft forever >> >> >> As you can see there is a /20 IPv4 allocated there. The agent.properties >> shows: >> >> host=10.100.1.6@static >> control.cidr=169.254.240.0/20 >> private.network.device=cloudbr1 >> guid=44e94fad-4aec-414e-8cac-cea3264dd431 >> >> Those are the relevant ones in this case. >> >> >> 6: enxb8599f206ba6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9216 qdisc mq >> state UP group default qlen 1000 >> link/ether b8:59:9f:20:6b:a6 brd ff:ff:ff:ff:ff:ff >> inet6 2a05:xxxx:601:117:ba59:9fff:fe20:6ba6/64 scope global deprecated >> dynamic mngtmpaddr >> valid_lft 3592sec preferred_lft 0sec >> inet6 fe80::ba59:9fff:fe20:6ba6/64 scope link >> valid_lft forever preferred_lft forever >> 7: enxb8599f206ba7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9216 qdisc mq >> state UP group default qlen 1000 >> link/ether b8:59:9f:20:6b:a7 brd ff:ff:ff:ff:ff:ff >> inet6 2a05:xxxx:601:197:ba59:9fff:fe20:6ba7/64 scope global deprecated >> dynamic mngtmpaddr >> valid_lft 3594sec preferred_lft 0sec >> inet6 fe80::ba59:9fff:fe20:6ba7/64 scope link >> valid_lft forever preferred_lft forever >> >> >> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group >> default qlen 1000 >> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 >> inet 127.0.0.1/8 scope host lo >> valid_lft forever preferred_lft forever >> inet 10.255.255.10/32 brd 10.255.255.10 scope global lo >> valid_lft forever preferred_lft forever >> inet6 2a05:xxxx:601:2::10/128 scope global >> valid_lft forever preferred_lft forever >> inet6 ::1/128 scope host >> valid_lft forever preferred_lft forever >> >> >> With two BGP sessions over these two uplinks: >> >> L2VPN EVPN Summary: >> BGP router identifier 10.255.255.10, local AS number 4200100124 VRF default >> vrf-id 0 >> BGP table version 0 >> RIB entries 4341, using 543 KiB of memory >> Peers 2, using 48 KiB of memory >> Peer groups 1, using 64 bytes of memory >> >> Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down >> State/PfxRcd PfxSnt Desc >> enxb8599f206ba6 4 4200100002 6027297 5839388 39828 0 0 28w1d10h >> 6635 6706 N/A >> enxb8599f206ba7 4 4200100002 6035334 5835598 39828 0 0 28w1d10h >> 6635 6706 N/A >> >> >> >> That's all! There is no cloudbr0 on the hypervisor at all. >> >> hv-138-a05-zz.xxxx.cldin.net# sh evpn vni >> VNI Type VxLAN IF # MACs # ARPs # Remote VTEPs >> Tenant VRF >> 539 L2 vxlan539 19 8 15 default >> 100 L2 vxlan100 107 106 103 default >> 528 L2 vxlan528 92 8 36 default >> 507 L2 vxlan507 43 8 22 default >> .... >> .... >> hv-138-a05-zz.xxxx.cldin.net# >> >> >> Wido >> >> On 4/4/25 04:07, Chi vediamo wrote: >>> I decided to reinstall everything clean again, drop the tables completely >>> and start all over again. >>> The management IPs I see them routed. even the ones for the s-VM and v- VM. >>> I see them on the transport switches. >>> The Public VXLAN is created, the Public IP address I am not able to see >>> them anywhere ! >>> FROM THE UI: If I run a diagnostic Ping to the public IP for the >>> secondarystoragevm or for the consoleproxy - success !!! >>> Why or how is not being routed ? How can I troubleshoot this? >>> vxlan1000: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9050 qdisc noqueue master >>> brvx-1000 state UNKNOWN group default qlen 1000 >>> link/ether 1a:00:21:de:dc:bc brd ff:ff:ff:ff:ff:ff >>> brvx-1000: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9100 qdisc noqueue state >>> UP group default qlen 1000 >>> link/ether 1a:00:21:de:dc:bc brd ff:ff:ff:ff:ff:ff >>> Any tip will be appreciated. >>> Thank you Teams. >>> Tata Y. >>>> On Apr 3, 2025, at 12:47 PM, Chi vediamo <tatay...@gmail.com> wrote: >>>> >>>> >>>> Using [Network 1] Only for all three [Management], [guest],[public] Tagged >>>> as VXLAN and with cloudbr1 >>>> >>>> *- **Should I use a separate Network for each ? ? *** >>>> >>>> What should I check, in addition to the official manual: I reread again >>>> the instructions and saw videos >>>> https://www.youtube.com/watch?v=9gXEmWbgX2o <<Actually this video shown >>>> both VMs are down' >>>> https://www.youtube.com/watch?v=DlJg3LYvIIs << this shows both on green. >>>> https://www.youtube.com/watch?v=vZfHfoYEMdY << this one shows the [Agent >>>> State] green while the Vms are in starting mode with a bue dot. >>>> I have Read the https://rohityadav.cloud/blog/cloudstack-kvm/ >>>> >>>> - Management servers are two physical separated servers. all Ubuntu 22.04 >>>> - DB is mysql running on a separate server. Mysql 8 >>>> - Hypervisor is running on a separate server. Ubuntu22.0 KVM >>>> - Cloudstack 4.20.0.0 >>>> - Primary Storage CEPH RDB >>>> - Secondary storage EMC NFS >>>> >>>> >>>> ***Issue 1) I can't add a second Management server, No idea why !!** * >>>> Thank you Wei, as soon i shut the second Management server the >>>> initial VMs proxy and secondary storage come up >>>> >>>> ***Issue 2) Management Network is Tagged as VXLAN, the UI shows >>>> vlan://untagged - "no typo vlan" every-time i use the UI to change to >>>> vxlan://untagged** * >>>> I did try vxlan://untagged parameter Got the message: Vlan parameter >>>> : vxlan://untagged is not in valid format >>>> I did try vxlan:// Got the message: >>>> Unable to convert to broadcast URI: vxlan:// >>>> I did try vxlan:// Got the message: >>>> Vlan parameter : vxlan://100 is not in valid format' >>>> >>>> ***Issue 3) s-VM and Proxy come up but Public IP addresses are not being >>>> routed Advertised.*** >>>> v-154-VM <public-IP-3> <Private-IP> <control-ip-169.x.x.x> >>>> [State-Running] [Agent State-Gray] >>>> s-285-VM <public-IP-2> <Private-IP> <control-ip-169.x.x.y> >>>> [State-Running] [Agent State-Gray] >>>> If both [State] and [Agent State] need to be green, What other debug >>>> can I enable ?? >>>> >>>> **NOTE** The VXLAN parameter for the Public network was taken >>>> vxlan://1000, But are not routed. >>>> >>>> I have to use a Single management server as everything messes UP when I >>>> run a second Management server with the Same Database. >>>> >>>> I reiterate The initial VMs are running: >>>> v-154-VM <public-IP-3> <Private-IP> <control-ip-169.x.x.x> >>>> s-285-VM <public-IP-2> <Private-IP> <control-ip-169.x.x.y> >>>> >>>> >>>> >>>> I do not see the Public IP anywhere, not even advertised on vni1000 or any >>>> other, the public VNI is there was created by the system, but nothing is >>>> there - No IP addresses advertised >>>> >>>> I used VLAN/VNI vxlan://1000 >>>> >>>> vxlan1000: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9050 master brvx-1000 >>>> state forwarding priority >>>> There are 2 VNETS assigned >>>> >>>> virsh domiflist s-285-VM >>>> Interface Type Source Model MAC >>>> -------------------------------------------------------------- >>>> vnet3 bridge cloud0 virtio 0e:00:a9:fe:a1:14 >>>> vnet4 bridge cloudbr1 virtio 1e:00:3f:00:03:33 >>>> vnet5 bridge brvx-1000 virtio 1e:00:b8:00:0c:fe >>>> >>>> virsh domifaddr s-285-VM >>>> Name MAC address Protocol Address >>>> ------------------------------------------------------------------------------- >>>> ----- Nothing here ----- >>>> >>>> >>>> the command arp -n, does not show Public IP addresses neither the >>>> management IP addresses, just the private IP addresses and the >>>> <control-ip-169.x.x.x> >>>> >>>> sh bgp evpn route vni 1000 >>>> BGP table version is 1, local router ID is 10.1.1.14 >>>> Status codes: s suppressed, d damped, h history, * valid, > best, i - >>>> internal >>>> Origin codes: i - IGP, e - EGP, ? - incomplete >>>> EVPN type-1 prefix: [1]:[EthTag]:[ESI]:[IPlen]:[VTEP-IP]:[Frag-id] >>>> EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP] >>>> EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP] >>>> EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP] >>>> EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP] >>>> >>>> Network Next Hop Metric LocPrf Weight Path >>>> *> [3]:[0]:[32]:[10.1.1.14] >>>> 10.1.1.14(hv1) >>>> 32768 i >>>> ET:8 RT:5641:1000 >>>> >>>> Displayed 1 prefixes (1 paths) >>>> >>>> At the Agent side only WARNINGS I am not using uefi. >>>> >>>> 2025-04-03 00:01:44,744 ERROR [kvm.resource.LibvirtComputingResource] >>>> (main:[]) (logid:) uefi properties file not found due to: Unable to find >>>> file uefi.properties. >>>> 2025-04-03 00:01:46,815 WARN [utils.script.Script] (Agent-Handler-1: []) >>>> (logid:) Process [15539] for command [/bin/bash -c virt-v2v -- version ] >>>> encountered the error: [127]. >>>> 2025-04-03 00:01:47,921 ERROR [cloud.agent.Agent] (agentRequest- >>>> Handler-5:[]) (logid:09929bd6) Unexpected arch null, expected x86_64 >>>> 2025-04-03 00:01:46,815 WARN [utils.script.Script] (Agent-Handler-1: []) >>>> (logid:) Execution of process [15539] for command [/bin/bash -c virt-v2v >>>> --version ] failed. >>>> 2025-04-03 00:01:46,815 WARN [utils.script.Script] (Agent-Handler-1: []) >>>> (logid:) Process [15539] for command [/bin/bash -c virt-v2v -- version ] >>>> encountered the error: [127]. >>>> >>>> >>>> AT the management the rror is: >>>> 2025-04-03 00:00:28,115 DEBUG [c.c.c.ClusterServiceServletImpl] >>>> (Cluster-Worker-5:[ctx-c96bca9e]) (logid:3fbff637) Executing >>>> ClusterServicePdu with service URL: https://10.1.1.1:9090/clusterservice >>>> 2025-04-03 00:21:28,122 ERROR [c.c.c.ClusterServiceServletImpl] >>>> (Cluster-Worker-5:[ctx-c96bca9e]) (logid:3fbff637) Exception from : >>>> https://10.1.1.1:9090/clusterservice, method : null, exception : >>>> [javax.net](http://javax.net/).ssl.SSLPeerUnverifiedException: Certificate >>>> for <10.1.1.1> doesn't match any of the subject alternative names: >>>> [fde0:f:2897:1:1:0:0:1, 172.1.1.1, fe80:0:0:0:e643:4bff:fe81:9660, >>>> cs1.myinternaldomain.int, cloudstack.internal] >>>> >>>> >>>> Tata Y. >> >
