Re: NIC / VLAN ISSUE ON KVM HOST

Fri, 15 Mar 2013 23:08:58 -0700 

Hi Valery,

here is the network layout I did (Advanced, shared network) :

KVM :

bond0  (also eth0 or any other Ethernet device works)
+
+----- bond0.x (x = Network VLANID ) -----> cloud-mgmt (Linux bridge)
+
+----- bond0.y (another VLAN) ------- > cloud-storage (Linux bridge)
+
+----- cloudbr0 (Linux Bridge, just a helper)


CSTK :
Management using traffic/KVM label : cloud-mgmt (the Linux interface name, untagged (no tag support anyway in CSTK)) 
Storage using traffic label : cloud-storage (untagged)
Guest Network using traffic label : cloudbr0 with VLANID A
Public Network using traffic label : cloudbr0 with VLANID B
The magic begins with cloudbr0, because CSTK finds out the physical interface (bond0 here) and creates new VLANs A and B 
to new bridges cloudVirBrA and cloudVirBrB.
Everything happens automatically so you only have to create the management ,storage and cloud0 bridges. 


Here a real "brctl show" sample from my network:

cloud-mgmt        8000.00219b937e8c     no bond0.2610
cloud-storage     8000.00219b937e8c     no bond0.2578
cloudVirBr2562    8000.00219b937e8c     no bond0.2562
cloudVirBr2615    8000.00219b937e8c     no bond0.2615
cloudbr0          8000.00219b937e8c     no bond0
If you cloud0 is a bridge which connects the local link interface of the VM with the hypervisor. Also 
automatically created by CSTK
You can also check out https://cwiki.apache.org/confluence/display/CLOUDSTACK/Cloud+Operators there are some nice examples shown for different network designs. If you are not familiar with all that routing and VLAN and bridging stuff I can send you some examples. 

Have fun,
Bjoern

On 3/14/2013 5:59 AM, Valery Fongang wrote:

Hi Bjoern,

Thanks for your reply (and sorry for my late response) but I would like more indication 
on how to do this: "Bridge on VLAN and don't use VLAN ID. That's the combination I 
choose because I had trouble with the first solution.".

My main goal is to implement Cloudstack with advanced networking with VLANs and 
I have a single NIC on my host with no intention of separating the Storage LAN. 
As you may have mentioned yes I have a physical network declarartion in 
CloudStack Management interface a range of VLAN specified. I'm actually not too 
sure of what I'm doing apart from trying to get all this to work with VLANs. I 
will be very happy if you can give me a detailed config example for my Host NIC 
and Network in general.




-----Original Message-----
From: Bjoern Teipel [mailto:bjoern.tei...@internetbrands.com]
Sent: 11 March 2013 19:36
To: Ahmad Emneina
Cc: Valery Fongang; Marcus Sorensen; Edison Su; Cloudstack users
Subject: Re: NIC / VLAN ISSUE ON KVM HOST

Looks like you specified a network in CS, probably the storage, using a VLAN ID 
(VID) but you have the vlan ID already bound to a cloudbr bridge. You can only 
do either one,

Bridge on native ETH/bond interface and use Vlan ID in CS. CS will do the rest. 
I had problems using a VID on the storage network

Or

Bridge on VLAN and don't use VLAN ID. That's the combination I choose because I 
had trouble with the first solution.

Guest and public networks worked fine for me using VID in CS.

Bjoern

Ahmad Emneina <aemne...@gmail.com> wrote:


adding the KVM gurus. Edison and Marcus. They are heavily active on the
dev list relating to KVM.


On Mon, Mar 11, 2013 at 3:16 AM, Valery Fongang <
fongang.val...@pertechtual.net> wrote:


Hi,

I have some issues sorting out how I need to set my NIC on my KVM
Host for CloudStack to work correctly. Here is my setup:

- 1 KVM (Centos 6.3) host with a single NIC and I am using
SharedMointPoint option for my Primary Storage to leverage Local
Storage on this server.
- 1 VM acting as CMS on a different Physical Server. NFS Secondary
Storage is shared from this VM.

My Network settings on the KVM host are as follow and at the bottom
my CMS log. How am I supposed to set my network to prevent having the
error in this log?

#######################Network Config ###########################

cloud0    Link encap:Ethernet  HWaddr 76:D4:8A:DB:9C:B3
           inet addr:169.254.0.1  Bcast:169.254.255.255  Mask:255.255.0.0
           inet6 addr: fe80::74d4:8aff:fedb:9cb3/64 Scope:Link
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
           TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:0
           RX bytes:0 (0.0 b)  TX bytes:468 (468.0 b)

cloudbr0  Link encap:Ethernet  HWaddr C8:60:00:58:9F:13
           inet addr:10.26.0.143  Bcast:10.26.255.255  Mask:255.255.0.0
           inet6 addr: fe80::ca60:ff:fe58:9f13/64 Scope:Link
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:3215 errors:0 dropped:0 overruns:0 frame:0
           TX packets:2148 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:0
           RX bytes:318720 (311.2 KiB)  TX bytes:1490094 (1.4 MiB)

cloudbr1  Link encap:Ethernet  HWaddr C8:60:00:58:9F:13
           inet6 addr: fe80::ca60:ff:fe58:9f13/64 Scope:Link
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
           TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:0
           RX bytes:0 (0.0 b)  TX bytes:578 (578.0 b)

eth0      Link encap:Ethernet  HWaddr C8:60:00:58:9F:13
           inet6 addr: fe80::ca60:ff:fe58:9f13/64 Scope:Link
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:5783 errors:0 dropped:0 overruns:0 frame:0
           TX packets:7097 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:1000
           RX bytes:640522 (625.5 KiB)  TX bytes:1838861 (1.7 MiB)
           Interrupt:34

eth0.1    Link encap:Ethernet  HWaddr C8:60:00:58:9F:13
           inet6 addr: fe80::ca60:ff:fe58:9f13/64 Scope:Link
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:4798 errors:0 dropped:0 overruns:0 frame:0
           TX packets:3310 errors:0 dropped:6 overruns:0 carrier:0
           collisions:0 txqueuelen:0
           RX bytes:514130 (502.0 KiB)  TX bytes:1551030 (1.4 MiB)

eth0.200  Link encap:Ethernet  HWaddr C8:60:00:58:9F:13
           inet6 addr: fe80::ca60:ff:fe58:9f13/64 Scope:Link
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
           TX packets:2601 errors:0 dropped:7 overruns:0 carrier:0
           collisions:0 txqueuelen:0
           RX bytes:0 (0.0 b)  TX bytes:225281 (220.0 KiB)

eth0.300  Link encap:Ethernet  HWaddr C8:60:00:58:9F:13
           inet6 addr: fe80::ca60:ff:fe58:9f13/64 Scope:Link
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
           TX packets:1179 errors:0 dropped:3 overruns:0 carrier:0
           collisions:0 txqueuelen:0
           RX bytes:0 (0.0 b)  TX bytes:61972 (60.5 KiB)

lo        Link encap:Local Loopback
           inet addr:127.0.0.1  Mask:255.0.0.0
           inet6 addr: ::1/128 Scope:Host
           UP LOOPBACK RUNNING  MTU:16436  Metric:1
           RX packets:2 errors:0 dropped:0 overruns:0 frame:0
           TX packets:2 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:0
           RX bytes:190 (190.0 b)  TX bytes:190 (190.0 b)

virbr0    Link encap:Ethernet  HWaddr 52:54:00:93:DC:18
           inet addr:192.168.122.1  Bcast:192.168.122.255
  Mask:255.255.255.0
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:0
           RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
###################### brctl show ###################### [root@kvm
home]# brctl show
bridge name     bridge id               STP enabled     interfaces
cloud0          8000.000000000000       no
cloudVirBr1             8000.000000000000       no
cloudbr0                8000.c86000589f13       yes             eth0.1
                                                         eth0.200
cloudbr1                8000.c86000589f13       yes             eth0.300
virbr0          8000.52540093dc18       yes             virbr0-nic


##################### Recent Error Log################

2013-03-11 09:56:17,421 DEBUG [cloud.alert.AlertManagerImpl]
(catalina-exec-11:null) Done executing storage capacity update
2013-03-11 09:56:17,421 DEBUG [cloud.alert.AlertManagerImpl]
(catalina-exec-11:null) Executing capacity updates for public ip and
Vlans
2013-03-11 09:56:17,429 DEBUG [cloud.alert.AlertManagerImpl]
(catalina-exec-11:null) Done capacity updates for public ip and Vlans
2013-03-11 09:56:17,429 DEBUG [cloud.alert.AlertManagerImpl]
(catalina-exec-11:null) Executing capacity updates for private ip
2013-03-11 09:56:17,432 DEBUG [cloud.alert.AlertManagerImpl]
(catalina-exec-11:null) Done executing capacity updates for private
ip
2013-03-11 09:56:17,432 DEBUG [cloud.alert.AlertManagerImpl]
(catalina-exec-11:null) Done recalculating system capacity
2013-03-11 09:56:21,888 DEBUG [cloud.server.StatsCollector]
(StatsCollector-2:null) VmStatsCollector is running...
2013-03-11 09:56:23,743 DEBUG [agent.transport.Request]
(AgentManager-Handler-5:null) Seq 5-1100559375: Processing:  { Ans: ,
MgmtId: 16279293006507, via: 5, Ver: v1, Flags: 110,
[{"StartAnswer":{"vm":{"id":16209,"name":"s-16209-VM","type":"Seconda
ryStorageVm","cpus":1,"speed":500,"minRam":268435456,"maxRam":2684354
56,"arch":"x86_64","os":"Debian GNU/Linux 5.0 (32-bit)","bootArgs":"
template=domP type=secstorage
host=10.26.0.90 port=8250 name=s-16209-VM zone=4 pod=4
guid=s-16209-VM
resource=com.cloud.storage.resource.PremiumSecondaryStorageResource
instance=SecStorage sslcopy=true role=templateProcessor mtu=1500
eth2ip=172.18.1.15 eth2mask=255.255.0.0 eth2mask=255.255.0.0
gateway=172.18.1.1 public.network.device=eth2 eth0ip=169.254.1.144
eth0mask=255.255.0.0 eth0mask=255.255.0.0 eth1ip=10.26.0.136
eth1mask=255.255.0.0 eth1mask=255.255.0.0
mgmtcidr=10.26.0.0/16localgw=10.26.0.1 private.network.device=eth1
eth3ip=10.26.0.134
eth3mask=255.255.0.0 eth3mask=255.255.0.0 storageip=10.26.0.134
storagenetmask=255.255.0.0 storagegateway=10.26.0.1
internaldns1=10.26.0.160 internaldns2=8.8.8.8 dns1=10.26.0.160
dns2=8.8.8.8","rebootOnCrash":false,"enableHA":false,"limitCpuUse":fa
lse,"vncPassword":"be60d6697a342397","params":{},"disks":[{"id":16209
,"name":"/home","mountPoint":"efd20cac-c832-4a57-88bf-ff939e25943b","
path":"efd20cac-c832-4a57-88bf-ff939e25943b","size":139264,"type":"RO
OT","storagePoolType":"SharedMountPoint","storagePoolUuid":"390801ff-
45b9-4177-9620-b863a3358b2d","deviceId":0}],"nics":[{"deviceId":2,"ne
tworkRateMbps":-1,"defaultNic":true,"ip":"172.18.1.15","netmask":"255
.255.0.0","gateway":"172.18.1.1","mac":"06:07:fc:00:00:11","dns1":"10
.26.0.160","dns2":"8.8.8.8","broadcastType":"Vlan","type":"Public","b
roadcastUri":"vlan://1","isolationUri":"vlan://1","isSecurityGroupEna
bled":false},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"i
p":"169.254.1.144","netmask":"255.255.0.0","gateway":"169.254.0.1","m
ac":"0e:00:a9:fe:01:90","broadcastType":"LinkLocal","type":"Control",
"isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"d
efaultNic":false,"ip":"10.26.0.136","netmask":"255.255.0.0","gateway"
:"10.26.0.1","mac":"06:2d:d8:00:00:07","broadcastType":"Native","type
":"Management","isSecurityGroupEnabled":false},{"deviceId":3,"network
RateMbps":-1,"defaultNic":false,"ip":"10.26.0.134","netmask":"255.255
.0.0","gateway":"10.26.0.1","mac":"06:a5:3e:00:00:05","broadcastType"
:"Native","type":"Storage","isSecurityGroupEnabled":false}]},"result"
:false,"details":"Failed to create vnet 1: Set name-type for VLAN
subsystem. Should be visible in /proc/net/vlan/configdevice eth0.1 is
already a member of a bridge; can't enslave it to bridge
cloudVirBr1.Failed to add vlan: eth0.1 to
cloudVirBr1","wait":0}},{"Answer":{"result":false,"details":"Stopped
by previous failure","wait":0}}] }
2013-03-11 09:56:23,743 DEBUG [agent.transport.Request]
(secstorage-1:null) Seq 5-1100559375: Received:  { Ans: , MgmtId:
16279293006507, via: 5, Ver: v1, Flags: 110, { StartAnswer, Answer }
}
2013-03-11 09:56:23,743 WARN  [cloud.vm.VirtualMachineManagerImpl]
(secstorage-1:null) Cleanup failed due to Failed to create vnet 1:
Set name-type for VLAN subsystem. Should be visible in
/proc/net/vlan/configdevice eth0.1 is already a member of a bridge;
can't enslave it to bridge cloudVirBr1.Failed to add vlan: eth0.1 to
cloudVirBr1
2013-03-11 09:56:23,743 WARN  [cloud.vm.VirtualMachineManagerImpl]
(secstorage-1:null) Cleanup failed due to Stopped by previous failure
2013-03-11 09:56:23,744 WARN  [cloud.vm.VirtualMachineManagerImpl]
(AgentManager-Handler-5:null) Cleanup failed due to Failed to create
vnet
1: Set name-type for VLAN subsystem. Should be visible in
/proc/net/vlan/configdevice eth0.1 is already a member of a bridge;
can't enslave it to bridge cloudVirBr1.Failed to add vlan: eth0.1 to
cloudVirBr1
2013-03-11 09:56:23,744 WARN  [cloud.vm.VirtualMachineManagerImpl]
(AgentManager-Handler-5:null) Cleanup failed due to Stopped by
previous failure
2013-03-11 09:56:23,744 DEBUG [agent.manager.AgentAttache]
(AgentManager-Handler-5:null) Seq 5-1100559376: Sending now.  is
current sequence.
2013-03-11 09:56:23,750 INFO  [cloud.vm.VirtualMachineManagerImpl]
(secstorage-1:null) Unable to start VM on Host[-5-Routing] due to
Failed to create vnet 1: Set name-type for VLAN subsystem. Should be
visible in /proc/net/vlan/configdevice eth0.1 is already a member of
a bridge; can't enslave it to bridge cloudVirBr1.Failed to add vlan:
eth0.1 to cloudVirBr1
2013-03-11 09:56:23,758 DEBUG [cloud.vm.VirtualMachineManagerImpl]
(secstorage-1:null) Cleaning up resources for the vm
VM[SecondaryStorageVm|s-16209-VM] in Starting state
2013-03-11 09:56:23,760 DEBUG [agent.transport.Request]
(secstorage-1:null) Seq 5-1100559384: Waiting for Seq 1100559376
Scheduling:  { Cmd , MgmtId: 16279293006507, via: 5, Ver: v1, Flags:
100111,
[{"StopCommand":{"isProxy":false,"vmName":"s-16209-VM","wait":0}}] }
2013-03-11 09:56:24,213 DEBUG
[network.router.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:null) Found 0 routers.
2013-03-11 09:56:25,364 DEBUG [agent.manager.AgentManagerImpl]
(AgentManager-Handler-6:null) Ping from 5
2013-03-11 09:56:37,004 DEBUG [cloud.server.StatsCollector]
(StatsCollector-2:null) HostStatsCollector is running...
2013-03-11 09:56:37,598 DEBUG [cloud.vm.VirtualMachineManagerImpl]
(AgentManager-Handler-7:null) Cleanup succeeded. Details null
2013-03-11 09:56:37,598 DEBUG [agent.transport.Request]
(StatsCollector-2:null) Seq 5-1100559385: Received:  { Ans: , MgmtId:
16279293006507, via: 5, Ver: v1, Flags: 10, { GetHostStatsAnswer } }

Reply via email to