Hi Serge, that configuration looks OK - cloudbr0 has to has an IP as you want to be able to have your host on the management network (and you removed the IP from the child-interface). cloudbr1 is use for public traffic, so that is also ok - NO IP should be there, as you do NOT want your host to be reachable via L3 on it's public IP address - only L2, pass through the traffic to the VMs.
In short, yes, it looks good. When in ACS, make sure to set "cloudbr0" as KVM traffic label for management/storage/guest traffic type, while "cloudbr1" (no quotes) will be the traffic label for public traffic. After all these changes, make sure to destroy SSVM/CPVM, and restart any networks/VPC with "cleanup" which will create new VRs attached to the correct bridges, etc. Good luck! On Mon, 24 May 2021 at 17:57, Serge Byishimo <sergebyish...@gmail.com> wrote: > Hello Mr Andrija, > > Thank you for your support, You are very right. > > I have two NICs on the Host enp0s25 and enp3s0, > > I plan to use > - cloubr0 for the management traffic and guest traffic > - cloubr1 for the public traffic > > Iβm going to label them accordingly in the physical network settings > > Before I continue, does this networking configuration look okay to you? > > [root@srvr2 network-scripts]# > [root@srvr2 network-scripts]# ll > total 16 > -rw-r--r--. 1 root root 160 May 21 15:16 ifcfg-cloudbr0 > -rw-r--r--. 1 root root 99 May 21 04:40 ifcfg-cloudbr1 > -rw-r--r--. 1 root root 105 May 24 11:12 ifcfg-enp0s25 > -rw-r--r--. 1 root root 135 May 24 11:47 ifcfg-enp3s0 > [root@srvr2 network-scripts]# > [root@srvr2 network-scripts]# > [root@srvr2 network-scripts]# > [root@srvr2 network-scripts]# cat ifcfg-cloudbr0 > DEVICE=cloudbr0 > TYPE=Bridge > ONBOOT=yes > BOOTPROTO=none > IPV6INIT=no > IPV6_AUTOCONF=no > DELAY=5 > IPADDR=192.168.1.7 > GATEWAY=192.168.1.1 > NETMASK=255.255.255.0 > STP=yes > [root@srvr2 network-scripts]# > [root@srvr2 network-scripts]# > [root@srvr2 network-scripts]# cat ifcfg-cloudbr1 > DEVICE=cloudbr1 > TYPE=Bridge > ONBOOT=yes > BOOTPROTO=none > IPV6INIT=no > IPV6_AUTOCONF=no > DELAY=5 > STP=yes > [root@srvr2 network-scripts]# > [root@srvr2 network-scripts]# > [root@srvr2 network-scripts]# cat ifcfg-enp0s25 > TYPE="Ethernet" > BOOTPROTO="none" > NAME="enp0s25" > DEVICE="enp0s25" > ONBOOT="yes" > BRIDGE=cloudbr0 > HOTPLUG=no > [root@srvr2 network-scripts]# > [root@srvr2 network-scripts]# > [root@srvr2 network-scripts]# cat ifcfg-enp3s0 > TYPE=Ethernet > NAME=enp3s0 > UUID=949aef09-7aee-492f-82c3-969e445b8746 > DEVICE=enp3s0 > ONBOOT=yes > HOTPLUG=no > BOOTPROTO=none > BRIDGE=cloudbr1 > [root@srvr2 network-scripts]# > [root@srvr2 network-scripts]# ifconfig > cloud0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > inet 169.254.0.1 netmask 255.255.0.0 broadcast 0.0.0.0 > inet6 fe80::5470:efff:fe14:35fa prefixlen 64 scopeid 0x20<link> > ether 56:70:ef:14:35:fa txqueuelen 1000 (Ethernet) > RX packets 0 bytes 0 (0.0 B) > RX errors 0 dropped 0 overruns 0 frame 0 > TX packets 13 bytes 1006 (1006.0 B) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > cloudbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > inet 192.168.1.7 netmask 255.255.255.0 broadcast 192.168.1.255 > inet6 fe80::c422:94ff:fe6a:bf92 prefixlen 64 scopeid 0x20<link> > ether d0:50:99:81:89:f5 txqueuelen 1000 (Ethernet) > RX packets 2519 bytes 264109 (257.9 KiB) > RX errors 0 dropped 147 overruns 0 frame 0 > TX packets 1842 bytes 235685 (230.1 KiB) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > cloudbr1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > inet6 fe80::c893:c7ff:fef7:8c8d prefixlen 64 scopeid 0x20<link> > ether 68:05:ca:c6:6e:9d txqueuelen 1000 (Ethernet) > RX packets 820 bytes 119498 (116.6 KiB) > RX errors 0 dropped 147 overruns 0 frame 0 > TX packets 12 bytes 980 (980.0 B) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > enp0s25: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > ether d0:50:99:81:89:f5 txqueuelen 1000 (Ethernet) > RX packets 3266 bytes 450429 (439.8 KiB) > RX errors 0 dropped 0 overruns 0 frame 0 > TX packets 1852 bytes 245701 (239.9 KiB) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > device interrupt 20 memory 0xfb200000-fb220000 > > enp3s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > ether 68:05:ca:c6:6e:9d txqueuelen 1000 (Ethernet) > RX packets 1300 bytes 258158 (252.1 KiB) > RX errors 0 dropped 0 overruns 0 frame 0 > TX packets 280 bytes 18180 (17.7 KiB) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > device interrupt 18 memory 0xfb1c0000-fb1e0000 > > lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 > inet 127.0.0.1 netmask 255.0.0.0 > inet6 ::1 prefixlen 128 scopeid 0x10<host> > loop txqueuelen 1000 (Local Loopback) > RX packets 8 bytes 718 (718.0 B) > RX errors 0 dropped 0 overruns 0 frame 0 > TX packets 8 bytes 718 (718.0 B) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > [root@srvr2 network-scripts]# > > Thank you > > > > On Mon, May 24, 2021 at 11:51 AM Andrija Panic <andrija.pa...@gmail.com> > wrote: > > > Hi Serge, > > > > please don't take me wrong, but you will have to go back to the drawing > > board in regards to networking - you seem to not understand how the Linux > > bridging works (or it may be just an oversee from your side) and this > > relates to CloudStack. > > > > 1. You have IP address set on both the network interface ifcfg-enp0s25 > and > > on the bridge ( ifcfg-cloudbr0) - the IP should only be set on bridge > level > > (remove all IP-related parameters from the interface that is joined to a > > bridge - see some example CloudStack docs on this) > > 2. Why do you have ifcfg-enp0s25.200 interface with cloudbr1 at all? What > > is this vlan 200 interface purpose? > > 3. What is the intention/purpose of each cloudb0 and cloudbr1 - what is > > each of these meant to carry, what kind of ACS traffic? > > 4. Do you have an understanding of what happens when i.e. ACS creates a > > network for a tenant (say vlan 150) - what does ACS does, how it reads > > traffic label to see which bridge to look for, extract its child > interface, > > then create VLAN 150 interface on top of that main interface, then > create a > > new ("empty") bridge and joins this VLAN 150 interface to this new > bridge, > > later joins VM's "vnet" interfaces to the same bridge, etc. > > > > Please do research a bit more, you seem to be missing a lot in network, > > and this is the usual place where people fail with ACS (where I also > failed > > many years ago - so nothing uncommon...) - see this blog page which > should > > explain a lot: > > - https://www.shapeblue.com/a-beginners-guide-to-cloudstack-networking/ > > - > > > > > https://www.shapeblue.com/understanding-cloudstacks-physical-networking-architecture/ > > > > I really hope that helps - please invest some time into understanding > this > > low-level, otherwise, your would be playing Russian roulette - it may to > > may not work... > > > > Cheers, > > > > On Mon, 24 May 2021 at 11:28, Serge Byishimo <sergebyish...@gmail.com> > > wrote: > > > > > I changed the traffic Label in the old UI for management, guest, > storage > > > and public network to cloudbr0 as this bridge exists in the host (I > have > > > one Host), as you can see in below images > > > > > > and then I restarted the System VMs, but I canβt telnet the Console > proxy > > > ports 80, 443 from the management server on Console Proxy Public IP > > address > > > > > > > > > Do I have to reinstall the whole set up using the old UI? > > > > > > [root@srvr1 ~]# > > > [root@srvr1 ~]# telnet 192.168.1.66 80 > > > Trying 192.168.1.66... > > > telnet: connect to address 192.168.1.66: No route to host > > > [root@srvr1 ~]# > > > [root@srvr1 ~]# > > > [root@srvr1 ~]# telnet 192.168.1.66 80 > > > Trying 192.168.1.66... > > > telnet: connect to address 192.168.1.66: No route to host > > > [root@srvr1 ~]# > > > [root@srvr1 ~]# telnet 192.168.1.66 443 > > > Trying 192.168.1.66... > > > telnet: connect to address 192.168.1.66: No route to host > > > [root@srvr1 ~]# > > > [root@srvr1 ~]# telnet 192.168.1.66 443 > > > Trying 192.168.1.66... > > > telnet: connect to address 192.168.1.66: No route to host > > > [root@srvr1 ~]# > > > > > > > > > > > > Management Traffic Labels > > > > > > https://ibb.co/R6M861X > > > > > > Guest Traffic Labels > > > > > > https://ibb.co/9ynMBm0 > > > > > > > > > Public Traffic Labels > > > > > > https://ibb.co/6YvjxgC > > > > > > Storage Traffic Labels > > > > > > https://ibb.co/99SMsT9 > > > > > > Physical Network > > > > > > https://ibb.co/TL6T5yZ > > > > > > ====== > > > > > > Host IFCONFIG > > > > > > > > > [root@srvr2 ~]# > > > [root@srvr2 ~]# ifconfig > > > br-200: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 > > > ether c2:0e:f3:be:8b:14 txqueuelen 1000 (Ethernet) > > > RX packets 0 bytes 0 (0.0 B) > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > TX packets 0 bytes 0 (0.0 B) > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > > > > > brenp0s25-200: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > > > inet6 fe80::fc9c:5dff:fe6d:cd99 prefixlen 64 scopeid > 0x20<link> > > > ether d0:50:99:81:89:f5 txqueuelen 1000 (Ethernet) > > > RX packets 153429 bytes 5240417 (4.9 MiB) > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > TX packets 81 bytes 5926 (5.7 KiB) > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > > > > > cloud0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > > > inet 169.254.0.1 netmask 255.255.0.0 broadcast 0.0.0.0 > > > inet6 fe80::c77:fbff:fe37:94cf prefixlen 64 scopeid > 0x20<link> > > > ether fe:00:a9:fe:4a:77 txqueuelen 1000 (Ethernet) > > > RX packets 51071 bytes 8524065 (8.1 MiB) > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > TX packets 59248 bytes 8380459 (7.9 MiB) > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > > > > > cloudbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > > > inet 192.168.1.7 netmask 255.255.255.0 broadcast > 192.168.1.255 > > > inet6 fe80::d250:99ff:fe81:89f5 prefixlen 64 scopeid > 0x20<link> > > > ether d0:50:99:81:89:f5 txqueuelen 1000 (Ethernet) > > > RX packets 1720903 bytes 13132311620 (12.2 GiB) > > > RX errors 0 dropped 67104 overruns 0 frame 0 > > > TX packets 1468695 bytes 6965166448 (6.4 GiB) > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > > > > > cloudbr1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 > > > ether 66:99:75:20:97:e0 txqueuelen 1000 (Ethernet) > > > RX packets 0 bytes 0 (0.0 B) > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > TX packets 0 bytes 0 (0.0 B) > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > > > > > enp0s25: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > > > ether d0:50:99:81:89:f5 txqueuelen 1000 (Ethernet) > > > RX packets 6212986 bytes 8178420327 (7.6 GiB) > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > TX packets 2912013 bytes 261573491 (249.4 MiB) > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > > device interrupt 20 memory 0xfb200000-fb220000 > > > > > > enp0s25.200: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > > > inet6 fe80::d250:99ff:fe81:89f5 prefixlen 64 scopeid > 0x20<link> > > > ether d0:50:99:81:89:f5 txqueuelen 1000 (Ethernet) > > > RX packets 0 bytes 0 (0.0 B) > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > TX packets 150299 bytes 6317510 (6.0 MiB) > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > > > > > enp3s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > > > inet6 fe80::6a05:caff:fec6:6e9d prefixlen 64 scopeid > 0x20<link> > > > ether 68:05:ca:c6:6e:9d txqueuelen 1000 (Ethernet) > > > RX packets 283323 bytes 37572267 (35.8 MiB) > > > RX errors 0 dropped 67126 overruns 0 frame 0 > > > TX packets 1877 bytes 175034 (170.9 KiB) > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > > device interrupt 18 memory 0xfb1c0000-fb1e0000 > > > > > > lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 > > > inet 127.0.0.1 netmask 255.0.0.0 > > > inet6 ::1 prefixlen 128 scopeid 0x10<host> > > > loop txqueuelen 1000 (Local Loopback) > > > RX packets 3018315 bytes 20666510791 (19.2 GiB) > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > TX packets 3018315 bytes 20666510791 (19.2 GiB) > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > > > > > vnet0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > > > inet6 fe80::fc00:a9ff:fefe:4a77 prefixlen 64 scopeid > 0x20<link> > > > ether fe:00:a9:fe:4a:77 txqueuelen 1000 (Ethernet) > > > RX packets 0 bytes 0 (0.0 B) > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > TX packets 16 bytes 1208 (1.1 KiB) > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > > > > > vnet1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > > > inet6 fe80::fc00:67ff:fe00:6 prefixlen 64 scopeid 0x20<link> > > > ether fe:00:67:00:00:06 txqueuelen 1000 (Ethernet) > > > RX packets 833 bytes 155599 (151.9 KiB) > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > TX packets 1500 bytes 221292 (216.1 KiB) > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > > > > > vnet2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > > > inet6 fe80::fc00:d6ff:fe00:11 prefixlen 64 scopeid 0x20<link> > > > ether fe:00:d6:00:00:11 txqueuelen 1000 (Ethernet) > > > RX packets 66 bytes 2772 (2.7 KiB) > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > TX packets 64 bytes 3148 (3.0 KiB) > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > > > > > vnet3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > > > inet6 fe80::fc00:a9ff:fefe:e245 prefixlen 64 scopeid > 0x20<link> > > > ether fe:00:a9:fe:e2:45 txqueuelen 1000 (Ethernet) > > > RX packets 161 bytes 19311 (18.8 KiB) > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > TX packets 206 bytes 16257 (15.8 KiB) > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > > > > > vnet4: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > > > inet6 fe80::fc00:48ff:fe00:7 prefixlen 64 scopeid 0x20<link> > > > ether fe:00:48:00:00:07 txqueuelen 1000 (Ethernet) > > > RX packets 352 bytes 39104 (38.1 KiB) > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > TX packets 1029 bytes 75691 (73.9 KiB) > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > > > > > vnet5: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > > > inet6 fe80::fc00:dbff:fe00:12 prefixlen 64 scopeid 0x20<link> > > > ether fe:00:db:00:00:12 txqueuelen 1000 (Ethernet) > > > RX packets 51 bytes 2142 (2.0 KiB) > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > TX packets 64 bytes 3148 (3.0 KiB) > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > > > > > [root@srvr2 ~]# > > > > > > > > > > > > Bridge Configuration > > > > > > [root@srvr2 network-scripts]# > > > [root@srvr2 network-scripts]# cat ifcfg-enp0s25 > > > TYPE="Ethernet" > > > BOOTPROTO="none" > > > NAME="enp0s25" > > > IPADDR="192.168.1.7" > > > NETMASK="255.255.255.0" > > > GATEWAY=192.168.1.1 > > > DNS=192.168.1.1 > > > DEVICE="enp0s25" > > > ONBOOT="yes" > > > BRIDGE=cloudbr0 > > > HOTPLUG=no > > > [root@srvr2 network-scripts]# > > > [root@srvr2 network-scripts]# > > > [root@srvr2 network-scripts]# > > > [root@srvr2 network-scripts]# cat ifcfg-cloudbr0 > > > DEVICE=cloudbr0 > > > TYPE=Bridge > > > ONBOOT=yes > > > BOOTPROTO=none > > > IPV6INIT=no > > > IPV6_AUTOCONF=no > > > DELAY=5 > > > IPADDR=192.168.1.7 > > > GATEWAY=192.168.1.1 > > > NETMASK=255.255.255.0 > > > STP=yes > > > [root@srvr2 network-scripts]# > > > > > > > > > [root@srvr2 network-scripts]# > > > [root@srvr2 network-scripts]# cat ifcfg-enp0s25.200 > > > DEVICE=enp0s25.200 > > > ONBOOT=yes > > > HOTPLUG=no > > > BOOTPROTO=none > > > TYPE=Ethernet > > > VLAN=yes > > > BRIDGE=cloudbr1 > > > [root@srvr2 network-scripts]# > > > > > > > > > > > > > > > On Mon, May 24, 2021 at 11:02 AM Andrija Panic < > andrija.pa...@gmail.com> > > > wrote: > > > > > > > Based on your screenshots, you have NOT set "Traffic Label" for each > > > > traffic type - this "traffic lab" should be == name of the bridge - > so > > > > CloudStack knows which bridge to use for specific VLAN interface > > > > provisioning etc - this is the reason why nothing works in your env, > > > > probably. > > > > > > > > Re the firewall, I guess reading the 2 following rules should tell > you > > > that > > > > Console traffic to port 80 and 443 is allowed: > > > > > > > > ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state > > NEW > > > > tcp > > > > dpt:443 > > > > ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state > > NEW > > > > tcp > > > > dpt:80 > > > > > > > > Go to old UI, and set traffic labels for Management, Storage, Guest > and > > > > Public traffic types to be exactly the same as the name of the > bridge > > > that > > > > you want to use for each traffic type (cloudb0 or cloudbr1 in your > > case - > > > > but you need to understand what you want to do, via which bridge, and > > the > > > > underlying VLAN physical interfaces on your hypervisors, and if your > > > > switches support trunking mode (to be able to send all VLANs between > > > > different hypervisor hosts etc) > > > > > > > > Best, > > > > > > > > On Mon, 24 May 2021 at 09:58, Serge Byishimo < > sergebyish...@gmail.com> > > > > wrote: > > > > > > > > > Hi Rohit, > > > > > > > > > > Please let me know how I should reset the network coz I did it > > multiple > > > > > times, > > > > > > > > > > The public traffic type VLAN/VNI is vlan://untagged as you can see > in > > > the > > > > > Public Traffic Type Image (https://ibb.co/pZyJ74x) , but while > > > > configuring > > > > > the Advanced Network Configuration they asked me for a VLAN / VNI > > range > > > > and > > > > > I Put 150-200 as you can see on the physical network 1 image ( > > > > > https://ibb.co/BKnD79B) > > > > > > > > > > > > > > > Zone Details > > > > > > > > > > https://ibb.co/RpHz3pG > > > > > > > > > > > > > > > Physical Network 1 > > > > > > > > > > https://ibb.co/BKnD79B > > > > > > > > > > > > > > > Guest Traffic Type > > > > > > > > > > https://ibb.co/JC7MRW9 > > > > > > > > > > Management Traffic Type > > > > > > > > > > https://ibb.co/2Pg67D8 > > > > > > > > > > Public Traffic Type > > > > > > > > > > https://ibb.co/pZyJ74x > > > > > > > > > > https://ibb.co/X5cn2qm > > > > > > > > > > > > > > > > > > > > Storage Traffic Type > > > > > > > > > > https://ibb.co/PrxJBfN > > > > > > > > > > Console Proxy Details > > > > > > > > > > https://ibb.co/qkHFS4D > > > > > > > > > > > > > > > Thank you, > > > > > > > > > > Kind Regards, > > > > > > > > > > > > > > > > > > > > On Mon, May 24, 2021 at 9:36 AM Rohit Yadav < > > rohit.ya...@shapeblue.com > > > > > > > > > wrote: > > > > > > > > > > > Hi Serge, > > > > > > > > > > > > Are you using VLAN 200 for your public IP range? You either need > to > > > fix > > > > > > your switches or remove the VLAN. Go to UI -> Zone -> physical > > > network > > > > -> > > > > > > and check the IP range for public management traffic type. If > > that's > > > > the > > > > > > case, disable the zone and then destroy your systemvms and then > > > remove > > > > > the > > > > > > IP range and readd the range with the VLAN untagged or blank. > > > > > > > > > > > > Regards. > > > > > > > > > > > > Regards, > > > > > > Rohit Yadav > > > > > > > > > > > > ________________________________ > > > > > > From: Serge Byishimo <sergebyish...@gmail.com> > > > > > > Sent: Monday, May 24, 2021 12:58:42 PM > > > > > > To: users@cloudstack.apache.org <users@cloudstack.apache.org> > > > > > > Subject: Please Help ππΌππΌ > > > > > > > > > > > > Centos8 > > > > > > Cloudstack 4.15 > > > > > > > > > > > > System VMs are running and their Agent is UP, > > > > > > > > > > > > System VMs can ping the management server but the management > server > > > can > > > > > not > > > > > > ping any of the IPs of the System VMs, thus the web console is > not > > > > > > accessible, > > > > > > > > > > > > Iβm thinking it is firewall rules on the System VM side that is > > > causing > > > > > it, > > > > > > > > > > > > Any reasons why this is happening? > > > > > > > > > > > > ======== > > > > > > > > > > > > Console Proxy > > > > > > > > > > > > > > > > > > root@v-2-VM:~# > > > > > > root@v-2-VM:~# > > > > > > root@v-2-VM:~# /usr/local/cloud/systemvm/ssvm-check.sh > > > > > > ================================================ > > > > > > First DNS server is 8.8.8.8 > > > > > > PING 8.8.8.8 (8.8.8.8): 56 data bytes > > > > > > 64 bytes from 8.8.8.8: icmp_seq=0 ttl=113 time=29.581 ms > > > > > > 64 bytes from 8.8.8.8: icmp_seq=1 ttl=113 time=30.140 ms > > > > > > --- 8.8.8.8 ping statistics --- > > > > > > 2 packets transmitted, 2 packets received, 0% packet loss > > > > > > round-trip min/avg/max/stddev = 29.581/29.861/30.140/0.280 ms > > > > > > Good: Can ping DNS server > > > > > > ================================================ > > > > > > Good: DNS resolves cloudstack.apache.org > > > > > > ================================================ > > > > > > ERROR: NFS is not currently mounted > > > > > > Try manually mounting from inside the VM > > > > > > NFS server is 255.255.255.0 > > > > > > PING 255.255.255.0 (255.255.255.0): 56 data bytes > > > > > > --- 255.255.255.0 ping statistics --- > > > > > > 2 packets transmitted, 0 packets received, 100% packet loss > > > > > > WARNING: cannot ping nfs server > > > > > > routing table follows > > > > > > Kernel IP routing table > > > > > > Destination Gateway Genmask Flags Metric Ref > > > Use > > > > > > Iface > > > > > > 0.0.0.0 192.168.1.1 0.0.0.0 UG 0 0 > > > 0 > > > > > > eth2 > > > > > > 8.8.4.4 192.168.1.1 255.255.255.255 UGH 0 0 > > > 0 > > > > > > eth1 > > > > > > 8.8.8.8 192.168.1.1 255.255.255.255 UGH 0 0 > > > 0 > > > > > > eth1 > > > > > > 169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 > > > 0 > > > > > > eth0 > > > > > > 192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 > > > 0 > > > > > > eth1 > > > > > > 192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 > > > 0 > > > > > > eth2 > > > > > > ================================================ > > > > > > Management server is 192.168.1.8. Checking connectivity. > > > > > > Good: Can connect to management server 192.168.1.8 port 8250 > > > > > > ================================================ > > > > > > Good: Java process is running > > > > > > ================================================ > > > > > > Tests Complete. Look for ERROR or WARNING above. > > > > > > root@v-2-VM:~# > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > root@v-2-VM:~# > > > > > > root@v-2-VM:~# iptables -nL > > > > > > Chain INPUT (policy DROP) > > > > > > target prot opt source destination > > > > > > ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 > > > > > > ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0 > > > > icmptype 8 > > > > > > ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 > > state > > > > > > RELATED,ESTABLISHED > > > > > > ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 > > state > > > > > > RELATED,ESTABLISHED > > > > > > ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 > > state > > > > > > RELATED,ESTABLISHED > > > > > > ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0 > > > > > > ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 > > state > > > > NEW > > > > > > tcp > > > > > > dpt:3922 > > > > > > ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 > > state > > > > NEW > > > > > > tcp > > > > > > dpt:8001 > > > > > > ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 > > state > > > > NEW > > > > > > tcp > > > > > > dpt:8001 > > > > > > ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 > > state > > > > NEW > > > > > > tcp > > > > > > dpt:443 > > > > > > ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 > > state > > > > NEW > > > > > > tcp > > > > > > dpt:80 > > > > > > ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 > > state > > > > NEW > > > > > > tcp > > > > > > dpt:8080 > > > > > > ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0 > > > > > > > > > > > > Chain FORWARD (policy DROP) > > > > > > target prot opt source destination > > > > > > > > > > > > Chain OUTPUT (policy ACCEPT) > > > > > > target prot opt source destination > > > > > > root@v-2-VM:~# > > > > > > root@v-2-VM:~# > > > > > > > > > > > > > > > > > > > > > > > > ================ > > > > > > > > > > > > Host Network Configuration > > > > > > > > > > > > > > > > > > [root@srvr2 byishimo]# > > > > > > [root@srvr2 byishimo]# ifconfig > > > > > > br-200: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 > > > > > > ether c2:0e:f3:be:8b:14 txqueuelen 1000 (Ethernet) > > > > > > RX packets 0 bytes 0 (0.0 B) > > > > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > > > > TX packets 0 bytes 0 (0.0 B) > > > > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions > 0 > > > > > > > > > > > > brenp0s25-200: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu > 1500 > > > > > > inet6 fe80::fc9c:5dff:fe6d:cd99 prefixlen 64 scopeid > > > > 0x20<link> > > > > > > ether d0:50:99:81:89:f5 txqueuelen 1000 (Ethernet) > > > > > > RX packets 151060 bytes 5159307 (4.9 MiB) > > > > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > > > > TX packets 77 bytes 5606 (5.4 KiB) > > > > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions > 0 > > > > > > > > > > > > cloud0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > > > > > > inet 169.254.0.1 netmask 255.255.0.0 broadcast 0.0.0.0 > > > > > > inet6 fe80::c77:fbff:fe37:94cf prefixlen 64 scopeid > > > > 0x20<link> > > > > > > ether fe:00:a9:fe:10:72 txqueuelen 1000 (Ethernet) > > > > > > RX packets 50364 bytes 8410715 (8.0 MiB) > > > > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > > > > TX packets 58434 bytes 8258165 (7.8 MiB) > > > > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions > 0 > > > > > > > > > > > > cloudbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > > > > > > inet 192.168.1.7 netmask 255.255.255.0 broadcast > > > > 192.168.1.255 > > > > > > inet6 fe80::d250:99ff:fe81:89f5 prefixlen 64 scopeid > > > > 0x20<link> > > > > > > ether d0:50:99:81:89:f5 txqueuelen 1000 (Ethernet) > > > > > > RX packets 1567172 bytes 12747824649 (11.8 GiB) > > > > > > RX errors 0 dropped 64945 overruns 0 frame 0 > > > > > > TX packets 1326109 bytes 6953807305 (6.4 GiB) > > > > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions > 0 > > > > > > > > > > > > cloudbr1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 > > > > > > ether 66:99:75:20:97:e0 txqueuelen 1000 (Ethernet) > > > > > > RX packets 0 bytes 0 (0.0 B) > > > > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > > > > TX packets 0 bytes 0 (0.0 B) > > > > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions > 0 > > > > > > > > > > > > enp0s25: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > > > > > > ether d0:50:99:81:89:f5 txqueuelen 1000 (Ethernet) > > > > > > RX packets 5929074 bytes 7781346038 (7.2 GiB) > > > > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > > > > TX packets 2762147 bytes 249304086 (237.7 MiB) > > > > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions > 0 > > > > > > device interrupt 20 memory 0xfb200000-fb220000 > > > > > > > > > > > > enp0s25.200: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > > > > > > inet6 fe80::d250:99ff:fe81:89f5 prefixlen 64 scopeid > > > > 0x20<link> > > > > > > ether d0:50:99:81:89:f5 txqueuelen 1000 (Ethernet) > > > > > > RX packets 0 bytes 0 (0.0 B) > > > > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > > > > TX packets 147973 bytes 6219514 (5.9 MiB) > > > > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions > 0 > > > > > > > > > > > > enp3s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > > > > > > inet6 fe80::6a05:caff:fec6:6e9d prefixlen 64 scopeid > > > > 0x20<link> > > > > > > ether 68:05:ca:c6:6e:9d txqueuelen 1000 (Ethernet) > > > > > > RX packets 277071 bytes 36652364 (34.9 MiB) > > > > > > RX errors 0 dropped 64965 overruns 0 frame 0 > > > > > > TX packets 1816 bytes 169340 (165.3 KiB) > > > > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions > 0 > > > > > > device interrupt 18 memory 0xfb1c0000-fb1e0000 > > > > > > > > > > > > lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 > > > > > > inet 127.0.0.1 netmask 255.0.0.0 > > > > > > inet6 ::1 prefixlen 128 scopeid 0x10<host> > > > > > > loop txqueuelen 1000 (Local Loopback) > > > > > > RX packets 2869120 bytes 18627106481 (17.3 GiB) > > > > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > > > > TX packets 2869120 bytes 18627106481 (17.3 GiB) > > > > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions > 0 > > > > > > > > > > > > vnet0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > > > > > > inet6 fe80::fc00:a9ff:fefe:1072 prefixlen 64 scopeid > > > > 0x20<link> > > > > > > ether fe:00:a9:fe:10:72 txqueuelen 1000 (Ethernet) > > > > > > RX packets 3020 bytes 433653 (423.4 KiB) > > > > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > > > > TX packets 4909 bytes 328353 (320.6 KiB) > > > > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions > 0 > > > > > > > > > > > > vnet1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > > > > > > inet6 fe80::fc00:58ff:fe00:c prefixlen 64 scopeid > > > 0x20<link> > > > > > > ether fe:00:58:00:00:0c txqueuelen 1000 (Ethernet) > > > > > > RX packets 35050 bytes 3465454 (3.3 MiB) > > > > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > > > > TX packets 91298 bytes 6368355 (6.0 MiB) > > > > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions > 0 > > > > > > > > > > > > vnet2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > > > > > > inet6 fe80::fc00:7dff:fe00:12 prefixlen 64 scopeid > > > 0x20<link> > > > > > > ether fe:00:7d:00:00:12 txqueuelen 1000 (Ethernet) > > > > > > RX packets 9891 bytes 572274 (558.8 KiB) > > > > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > > > > TX packets 68494 bytes 28588262 (27.2 MiB) > > > > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions > 0 > > > > > > > > > > > > vnet3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > > > > > > inet6 fe80::fc00:a9ff:fefe:d5d5 prefixlen 64 scopeid > > > > 0x20<link> > > > > > > ether fe:00:a9:fe:d5:d5 txqueuelen 1000 (Ethernet) > > > > > > RX packets 237 bytes 34056 (33.2 KiB) > > > > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > > > > TX packets 1236 bytes 75604 (73.8 KiB) > > > > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions > 0 > > > > > > > > > > > > vnet4: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > > > > > > inet6 fe80::fc00:46ff:fe00:8 prefixlen 64 scopeid > > > 0x20<link> > > > > > > ether fe:00:46:00:00:08 txqueuelen 1000 (Ethernet) > > > > > > RX packets 82835 bytes 3239704935 (3.0 GiB) > > > > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > > > > TX packets 129702 bytes 1948268672 (1.8 GiB) > > > > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions > 0 > > > > > > > > > > > > vnet5: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > > > > > > inet6 fe80::fc00:5aff:fe00:11 prefixlen 64 scopeid > > > 0x20<link> > > > > > > ether fe:00:5a:00:00:11 txqueuelen 1000 (Ethernet) > > > > > > RX packets 333471 bytes 19401455 (18.5 MiB) > > > > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > > > > TX packets 573568 bytes 1495800481 (1.3 GiB) > > > > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions > 0 > > > > > > > > > > > > vnet6: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > > > > > > inet6 fe80::fc00:5dff:fe00:22 prefixlen 64 scopeid > > > 0x20<link> > > > > > > ether fe:00:5d:00:00:22 txqueuelen 1000 (Ethernet) > > > > > > RX packets 36551 bytes 1535142 (1.4 MiB) > > > > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > > > > TX packets 44 bytes 3176 (3.1 KiB) > > > > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions > 0 > > > > > > > > > > > > vnet7: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > > > > > > inet6 fe80::fc00:a9ff:fefe:c3e1 prefixlen 64 scopeid > > > > 0x20<link> > > > > > > ether fe:00:a9:fe:c3:e1 txqueuelen 1000 (Ethernet) > > > > > > RX packets 10143 bytes 1997181 (1.9 MiB) > > > > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > > > > TX packets 11325 bytes 1899903 (1.8 MiB) > > > > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions > 0 > > > > > > > > > > > > vnet8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > > > > > > inet6 fe80::fc00:12ff:fe00:21 prefixlen 64 scopeid > > > 0x20<link> > > > > > > ether fe:00:12:00:00:21 txqueuelen 1000 (Ethernet) > > > > > > RX packets 909 bytes 298419 (291.4 KiB) > > > > > > RX errors 0 dropped 0 overruns 0 frame 0 > > > > > > TX packets 36585 bytes 1537926 (1.4 MiB) > > > > > > TX errors 0 dropped 0 overruns 0 carrier 0 collisions > 0 > > > > > > > > > > > > [root@srvr2 byishimo]# > > > > > > [root@srvr2 byishimo]# > > > > > > [root@srvr2 byishimo]# > > > > > > [root@srvr2 byishimo]# route -n > > > > > > Kernel IP routing table > > > > > > Destination Gateway Genmask Flags Metric Ref > > > Use > > > > > > Iface > > > > > > 0.0.0.0 192.168.1.1 0.0.0.0 UG 425 0 > > > 0 > > > > > > cloudbr0 > > > > > > 169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 > > > 0 > > > > > > cloud0 > > > > > > 192.168.1.0 0.0.0.0 255.255.255.0 U 425 0 > > > 0 > > > > > > cloudbr0 > > > > > > [root@srvr2 byishimo]# > > > > > > [root@srvr2 byishimo]# > > > > > > [root@srvr2 byishimo]# ip link list > > > > > > 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state > UNKNOWN > > > > mode > > > > > > DEFAULT group default qlen 1000 > > > > > > link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 > > > > > > 2: enp0s25: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc > > fq_codel > > > > > > master cloudbr0 state UP mode DEFAULT group default qlen 1000 > > > > > > link/ether d0:50:99:81:89:f5 brd ff:ff:ff:ff:ff:ff > > > > > > 3: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc > > fq_codel > > > > > state > > > > > > UP mode DEFAULT group default qlen 1000 > > > > > > link/ether 68:05:ca:c6:6e:9d brd ff:ff:ff:ff:ff:ff > > > > > > 4: cloudbr1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc > > > noqueue > > > > > > state DOWN mode DEFAULT group default qlen 1000 > > > > > > link/ether 66:99:75:20:97:e0 brd ff:ff:ff:ff:ff:ff > > > > > > 5: cloudbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc > > noqueue > > > > > state > > > > > > UP mode DEFAULT group default qlen 1000 > > > > > > link/ether d0:50:99:81:89:f5 brd ff:ff:ff:ff:ff:ff > > > > > > 6: br-200: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc > > noqueue > > > > > state > > > > > > DOWN mode DEFAULT group default qlen 1000 > > > > > > link/ether c2:0e:f3:be:8b:14 brd ff:ff:ff:ff:ff:ff > > > > > > 7: cloud0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc > noqueue > > > > state > > > > > > UP mode DEFAULT group default qlen 1000 > > > > > > link/ether fe:00:a9:fe:10:72 brd ff:ff:ff:ff:ff:ff > > > > > > 40: enp0s25.200@enp0s25: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu > > 1500 > > > > > qdisc > > > > > > noqueue master brenp0s25-200 state UP mode DEFAULT group default > > qlen > > > > > 1000 > > > > > > link/ether d0:50:99:81:89:f5 brd ff:ff:ff:ff:ff:ff > > > > > > 41: brenp0s25-200: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 > qdisc > > > > > noqueue > > > > > > state UP mode DEFAULT group default qlen 1000 > > > > > > link/ether d0:50:99:81:89:f5 brd ff:ff:ff:ff:ff:ff > > > > > > 84: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc > > fq_codel > > > > > master > > > > > > cloud0 state UNKNOWN mode DEFAULT group default qlen 1000 > > > > > > link/ether fe:00:a9:fe:10:72 brd ff:ff:ff:ff:ff:ff > > > > > > 85: vnet1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc > > fq_codel > > > > > master > > > > > > cloudbr0 state UNKNOWN mode DEFAULT group default qlen 1000 > > > > > > link/ether fe:00:58:00:00:0c brd ff:ff:ff:ff:ff:ff > > > > > > 86: vnet2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc > > fq_codel > > > > > master > > > > > > cloudbr0 state UNKNOWN mode DEFAULT group default qlen 1000 > > > > > > link/ether fe:00:7d:00:00:12 brd ff:ff:ff:ff:ff:ff > > > > > > 87: vnet3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc > > fq_codel > > > > > master > > > > > > cloud0 state UNKNOWN mode DEFAULT group default qlen 1000 > > > > > > link/ether fe:00:a9:fe:d5:d5 brd ff:ff:ff:ff:ff:ff > > > > > > 88: vnet4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc > > fq_codel > > > > > master > > > > > > cloudbr0 state UNKNOWN mode DEFAULT group default qlen 1000 > > > > > > link/ether fe:00:46:00:00:08 brd ff:ff:ff:ff:ff:ff > > > > > > 89: vnet5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc > > fq_codel > > > > > master > > > > > > cloudbr0 state UNKNOWN mode DEFAULT group default qlen 1000 > > > > > > link/ether fe:00:5a:00:00:11 brd ff:ff:ff:ff:ff:ff > > > > > > 90: vnet6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc htb > > > master > > > > > > brenp0s25-200 state UNKNOWN mode DEFAULT group default qlen 1000 > > > > > > link/ether fe:00:5d:00:00:22 brd ff:ff:ff:ff:ff:ff > > > > > > 91: vnet7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc > > fq_codel > > > > > master > > > > > > cloud0 state UNKNOWN mode DEFAULT group default qlen 1000 > > > > > > link/ether fe:00:a9:fe:c3:e1 brd ff:ff:ff:ff:ff:ff > > > > > > 92: vnet8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc htb > > > master > > > > > > brenp0s25-200 state UNKNOWN mode DEFAULT group default qlen 1000 > > > > > > link/ether fe:00:12:00:00:21 brd ff:ff:ff:ff:ff:ff > > > > > > [root@srvr2 byishimo]# > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Byishimo Shema Serge > > > > > Kigali, Rwanda > > > > > Tel: +250 786 076 106 > > > > > www.serge.works > > > > > > > > > > > > > > > > > -- > > > > > > > > Andrija PaniΔ > > > > > > > > > > > > > -- > > > Byishimo Shema Serge > > > Kigali, Rwanda > > > Tel: +250 786 076 106 > > > www.serge.works > > > > > > > > > -- > > > > Andrija PaniΔ > > > > > -- > Byishimo Shema Serge > Kigali, Rwanda > Tel: +250 786 076 106 > www.serge.works > -- Andrija PaniΔ