Re: [ovirt-users] LACP Bonding issue

[Derek Atkins]

Bryan Sockel  writes:

> That was my next thought, wanted to see if there was another way before I got
> to that point.

Just let me know if you want to see my configuration files in
/etc/sysconfig/network-scripts/  I'm happy to share them.

-derek
-- 
   Derek Atkins 617-623-3745
   de...@ihtfp.com www.ihtfp.com
   Computer and Internet Security Consultant
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] LACP Bonding issue

[Bryan Sockel]

That was my next thought, wanted to see if there was another way before
I got to that point.


 Original message 
From: Derek Atkins <de...@ihtfp.com> 
Date: 4/20/17 8:12 PM (GMT-06:00) 
To: Bryan Sockel <bryan.soc...@altn.com> 
Cc: Chris Adams <c...@cmadams.net>, users@ovirt.org 
Subject: Re: [ovirt-users] LACP Bonding issue 

  _  

>From : Derek Atkins [de...@ihtfp.com]
To : Bryan Sockel [bryan.soc...@altn.com]
Cc : Chris Adams [c...@cmadams.net], users@ovirt.org [users@ovirt.org]
Date : Thursday, April 20 2017 20:11:46
For what it's worth I set up my bond0 manually on CentOS before
installing
ovirt hosted engine and the ovirtmgmt bridge "took over" from bond0..
But
it appears to still be working.  At least I've not noticed links being
down, and "ifconfig" shows decent amount of traffic on both eno1 and
eno2.

Maybe wipe, re-install, configure it manually, and then install ovirt?

-derek

On Thu, April 20, 2017 8:58 pm, Bryan Sockel wrote:
> We checked the port groups, and servers are cabled correctly.
>
> After server is rebooted, em1 is the only interface passing traffic.
> Other 3 nics sitting idle.  We can down each port on the switch and
> confirm it is down on the server.
>
>
> I am pretty sure it is related to the bridge that was created to pass
> vm-host-altn traffic when the appliance was first installed.
>
>
>
>  Original message 
> From: Chris Adams 
> Date: 4/20/17 5:40 PM (GMT-06:00)
> To: users@ovirt.org
> Subject: Re: [ovirt-users] LACP Bonding issue
>
>   _
>
> From : Chris Adams [c...@cmadams.net]
> To : users@ovirt.org [users@ovirt.org]
> Date : Thursday, April 20 2017 17:40:25
> Once upon a time, Bryan Sockel  said:
>> It seems that is some disconnect between my network bridge, the bond
> and my
>> interfaces.  I would like to some how get my bond to use all 4
> interfaces.
>> On reboot, it always seems to reset consistently to EM1.
>
> Are you sure the switch side is all the same LACP group?  Sounds like
> one port may accidentally be in a separate group, and that happens to
be
> em1.
>
> You might try swapping wires between em1 and another port and reboot
and
> see which ports come up - if all but the port with the wire formerly
in
> em1 come up, it points to the switch config.
>
> --
> Chris Adams
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>


-- 
   Derek Atkins 617-623-3745
   de...@ihtfp.com www.ihtfp.com
   Computer and Internet Security Consultant

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] LACP Bonding issue

[Derek Atkins]

For what it's worth I set up my bond0 manually on CentOS before installing
ovirt hosted engine and the ovirtmgmt bridge "took over" from bond0..  But
it appears to still be working.  At least I've not noticed links being
down, and "ifconfig" shows decent amount of traffic on both eno1 and eno2.

Maybe wipe, re-install, configure it manually, and then install ovirt?

-derek

On Thu, April 20, 2017 8:58 pm, Bryan Sockel wrote:
> We checked the port groups, and servers are cabled correctly.
>
> After server is rebooted, em1 is the only interface passing traffic.
> Other 3 nics sitting idle.  We can down each port on the switch and
> confirm it is down on the server.
>
>
> I am pretty sure it is related to the bridge that was created to pass
> vm-host-altn traffic when the appliance was first installed.
>
>
>
>  Original message 
> From: Chris Adams <c...@cmadams.net>
> Date: 4/20/17 5:40 PM (GMT-06:00)
> To: users@ovirt.org
> Subject: Re: [ovirt-users] LACP Bonding issue
>
>   _
>
> From : Chris Adams [c...@cmadams.net]
> To : users@ovirt.org [users@ovirt.org]
> Date : Thursday, April 20 2017 17:40:25
> Once upon a time, Bryan Sockel  said:
>> It seems that is some disconnect between my network bridge, the bond
> and my
>> interfaces.  I would like to some how get my bond to use all 4
> interfaces.
>> On reboot, it always seems to reset consistently to EM1.
>
> Are you sure the switch side is all the same LACP group?  Sounds like
> one port may accidentally be in a separate group, and that happens to be
> em1.
>
> You might try swapping wires between em1 and another port and reboot and
> see which ports come up - if all but the port with the wire formerly in
> em1 come up, it points to the switch config.
>
> --
> Chris Adams
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>


-- 
   Derek Atkins 617-623-3745
   de...@ihtfp.com www.ihtfp.com
   Computer and Internet Security Consultant

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] LACP Bonding issue

[Chris Adams]

Sorry about the message with nothing new...

Once upon a time, Bryan Sockel  said:
> We checked the port groups, and servers are cabled correctly.
> 
> After server is rebooted, em1 is the only interface passing traffic.
> Other 3 nics sitting idle.  We can down each port on the switch and
> confirm it is down on the server.
> 
> I am pretty sure it is related to the bridge that was created to pass
> vm-host-altn traffic when the appliance was first installed.

Well, I don't have any problem with that setup on multiple oVirt
clusters (including a bunch of R610 servers), so I don't think that's
it.

I configure oVirt for "custom" bonding options; I use:

  mode=802.3ad lacp_rate=1 xmit_hash_policy=layer2+3

Is it possible to move the wires around temporarily, so different server
ports are connected to different switch ports?  It would be interested
to see if the "solo" behavior stayed with the port or the wire.

-- 
Chris Adams 
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] LACP Bonding issue

[Chris Adams]

Once upon a time, Bryan Sockel <bryan.soc...@altn.com> said:
> We checked the port groups, and servers are cabled correctly.
> 
> After server is rebooted, em1 is the only interface passing traffic.
> Other 3 nics sitting idle.  We can down each port on the switch and
> confirm it is down on the server.
> 
> 
> I am pretty sure it is related to the bridge that was created to pass
> vm-host-altn traffic when the appliance was first installed.
> 
> 
> 
>  Original message 
> From: Chris Adams <c...@cmadams.net> 
> Date: 4/20/17 5:40 PM (GMT-06:00) 
> To: users@ovirt.org 
> Subject: Re: [ovirt-users] LACP Bonding issue 
> 
>   _  
> 
> >From : Chris Adams [c...@cmadams.net]
> To : users@ovirt.org [users@ovirt.org]
> Date : Thursday, April 20 2017 17:40:25
> Once upon a time, Bryan Sockel  said:
> > It seems that is some disconnect between my network bridge, the bond
> and my 
> > interfaces.  I would like to some how get my bond to use all 4
> interfaces.  
> > On reboot, it always seems to reset consistently to EM1.
> 
> Are you sure the switch side is all the same LACP group?  Sounds like
> one port may accidentally be in a separate group, and that happens to be
> em1.
> 
> You might try swapping wires between em1 and another port and reboot and
> see which ports come up - if all but the port with the wire formerly in
> em1 come up, it points to the switch config.
> 
> -- 
> Chris Adams 
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users

-- 
Chris Adams <c...@cmadams.net>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] LACP Bonding issue

[Bryan Sockel]

We checked the port groups, and servers are cabled correctly.

After server is rebooted, em1 is the only interface passing traffic.
Other 3 nics sitting idle.  We can down each port on the switch and
confirm it is down on the server.


I am pretty sure it is related to the bridge that was created to pass
vm-host-altn traffic when the appliance was first installed.



 Original message 
From: Chris Adams <c...@cmadams.net> 
Date: 4/20/17 5:40 PM (GMT-06:00) 
To: users@ovirt.org 
Subject: Re: [ovirt-users] LACP Bonding issue 

  _  

>From : Chris Adams [c...@cmadams.net]
To : users@ovirt.org [users@ovirt.org]
Date : Thursday, April 20 2017 17:40:25
Once upon a time, Bryan Sockel  said:
> It seems that is some disconnect between my network bridge, the bond
and my 
> interfaces.  I would like to some how get my bond to use all 4
interfaces.  
> On reboot, it always seems to reset consistently to EM1.

Are you sure the switch side is all the same LACP group?  Sounds like
one port may accidentally be in a separate group, and that happens to be
em1.

You might try swapping wires between em1 and another port and reboot and
see which ports come up - if all but the port with the wire formerly in
em1 come up, it points to the switch config.

-- 
Chris Adams 
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] LACP Bonding issue

[Chris Adams]

Once upon a time, Bryan Sockel  said:
> It seems that is some disconnect between my network bridge, the bond and my 
> interfaces.  I would like to some how get my bond to use all 4 interfaces.  
> On reboot, it always seems to reset consistently to EM1.

Are you sure the switch side is all the same LACP group?  Sounds like
one port may accidentally be in a separate group, and that happens to be
em1.

You might try swapping wires between em1 and another port and reboot and
see which ports come up - if all but the port with the wire formerly in
em1 come up, it points to the switch config.

-- 
Chris Adams 
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] LACP Bonding issue

[Bryan Sockel]

After further testing we can either get EM1 running on the bond or EM2 - EM4 
running in the bond, but not EM1-EM4.  We are running the Hosted Engine 
Appliance in our setup.  In order to install the appliance, i have to point 
it to a physical nic during the install process.  The install process 
creates the network bridge ovirtmgmt which is tied to interface EM1 on both 
servers.  After i finished setting up the appliance, and both of my hosts, i 
went in and created my bond so i could configure LACP.  

It seems that is some disconnect between my network bridge, the bond and my 
interfaces.  I would like to some how get my bond to use all 4 interfaces.  
On reboot, it always seems to reset consistently to EM1.

-Original Message-
From: Konstantin Shalygin <k0...@k0ste.ru>
To: users@ovirt.org, Bryan Sockel <bryan.soc...@altn.com>
Date: Thu, 20 Apr 2017 23:43:15 +0700
Subject: Re: Re: [ovirt-users] LACP Bonding issue

You should configure your LAG with this options (custom mode on oVirt):

mode=4 miimon=100 xmit_hash_policy=2 lacp_rate=1

An tell to your network admin configure switch:
"Give me lacp timeout short with channel-group mode active. Also set 
port-channel load-balance src-dst-mac-ip (or src-dst-ip\src-dst-mac)".

You also need to understand that LACP balancing works 'per flow'. You can 
take 2 hosts and run "iperf -c xxx.xxx.xxx.xxx -i 0.1 -d",
and on one phy interface you should see 1Gb RX, and on another phy interface 
1Gb TX.

Hi,

I discovered an issue with my LACP configuration and i am having trouble 
figuring it out.  I am running 2 Dell Powered 610's with 4 broadcomm nics.  
I am trying to bond them together, however only one of the nics goes active 
no mater how much traffic i push across the links.

I have spoken to my network admin, and says that the switch ports are 
configured and can only see one active link on the switch.

Thanks
Bryan___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] LACP Bonding issue

[Konstantin Shalygin]

You should configure your LAG with this options (custom mode on oVirt):

mode=4 miimon=100 xmit_hash_policy=2 lacp_rate=1

An tell to your network admin configure switch:
"Give me lacp timeout short with channel-group mode active. Also set 
port-channel load-balance src-dst-mac-ip (or src-dst-ip\src-dst-mac)".


You also need to understand that LACP balancing works 'per flow'. You 
can take 2 hosts and run "iperf -c xxx.xxx.xxx.xxx -i 0.1 -d",
and on one phy interface you should see 1Gb RX, and on another phy 
interface 1Gb TX.



Hi,

I discovered an issue with my LACP configuration and i am having trouble
figuring it out.  I am running 2 Dell Powered 610's with 4 broadcomm nics.
I am trying to bond them together, however only one of the nics goes active
no mater how much traffic i push across the links.

I have spoken to my network admin, and says that the switch ports are
configured and can only see one active link on the switch.

Thanks
Bryan


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] LACP Bonding issue

[Bryan Sockel]

Hi,

I discovered an issue with my LACP configuration and i am having trouble 
figuring it out.  I am running 2 Dell Powered 610's with 4 broadcomm nics.  
I am trying to bond them together, however only one of the nics goes active 
no mater how much traffic i push across the links.

I have spoken to my network admin, and says that the switch ports are 
configured and can only see one active link on the switch.

Thanks
Bryan

Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2+3 (2)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

802.3ad info
LACP rate: slow
Min links: 0
Aggregator selection policy (ad_select): stable
System priority: 65535
System MAC address: f4:8e:38:c5:fc:a8
Active Aggregator Info:
Aggregator ID: 1
Number of ports: 1
Actor Key: 9
Partner Key: 20
Partner Mac Address: a4:6c:2a:e5:30:00

Slave Interface: em1
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: f4:8e:38:c5:fc:a8
Slave queue ID: 0
Aggregator ID: 1
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 0
Partner Churned Count: 0
details actor lacp pdu:
system priority: 65535
system mac address: f4:8e:38:c5:fc:a8
port key: 9
port priority: 255
port number: 1
port state: 61
details partner lacp pdu:
system priority: 8192
system mac address: a4:6c:2a:e5:30:00
oper key: 20
port priority: 32768
port number: 25
port state: 61

Slave Interface: em2
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: f4:8e:38:c5:fc:a9
Slave queue ID: 0
Aggregator ID: 2
Actor Churn State: churned
Actor Churned Count: 1
Partner Churned Count: 1
details actor lacp pdu:
system priority: 65535
system mac address: f4:8e:38:c5:fc:a8
port key: 9
port priority: 255
port number: 2
port state: 5
details partner lacp pdu:
system priority: 32768
system mac address: a4:6c:2a:e5:30:00
oper key: 20
port priority: 32768
port number: 73
port state: 5

Slave Interface: em3
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: f4:8e:38:c5:fc:aa
Slave queue ID: 0
Aggregator ID: 2
Actor Churn State: churned
Partner Churn State: churned
Actor Churned Count: 1
Partner Churned Count: 1
details actor lacp pdu:
system priority: 65535
system mac address: f4:8e:38:c5:fc:a8
port key: 9
port priority: 255
port number: 3
port state: 5
details partner lacp pdu:
system priority: 32768
system mac address: a4:6c:2a:e5:30:00
oper key: 20
port priority: 32768
port number: 26
port state: 5

Slave Interface: em4
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: f4:8e:38:c5:fc:ab
Slave queue ID: 0
Aggregator ID: 2
Actor Churn State: churned
Partner Churn State: churned
Actor Churned Count: 1
Partner Churned Count: 1
details actor lacp pdu:
oper key: 20
port priority: 32768
port number: 73
port state: 5

Slave Interface: em3
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: f4:8e:38:c5:fc:aa
Slave queue ID: 0
Aggregator ID: 2
Actor Churn State: churned
Partner Churn State: churned
Actor Churned Count: 1
Partner Churned Count: 1
details actor lacp pdu:
system priority: 65535
system mac address: f4:8e:38:c5:fc:a8
port key: 9
port priority: 255
port number: 3
port state: 5
details partner lacp pdu:
system priority: 32768
system mac address: a4:6c:2a:e5:30:00
oper key: 20
port priority: 32768
port number: 26
port state: 5

Slave Interface: em4
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: f4:8e:38:c5:fc:ab
Slave queue ID: 0
Aggregator ID: 2
Actor Churn State: churned
Partner Churn State: churned
Actor Churned Count: 1
Partner Churned Count: 1
details actor lacp pdu:
system priority: 65535
system mac address: f4:8e:38:c5:fc:a8
port key: 9
port priority: 255
port number: 4
port state: 5
details partner lacp pdu:
system priority: 32768
system mac address: a4:6c:2a:e5:30:00
oper key: 20
port priority: 32768
port number: 74
port state: 5
Apr 20 10:51:04 vm-host-colo-2 systemd: Stopping LSB: Bring up/down 
networking...
Apr 20 10:51:04 vm-host-colo-2 kernel: DMZ: port 1(bond0.10) entered disabled 
state
Apr 20 10:51:04 vm-host-colo-2 network: Shutting down interface DMZ:  [  OK  ]
Apr 20 10:51:04 vm-host-colo-2 kernel: Internal-Dev: port 1(bond0.30) entered 
disabled state
Apr 20 10:51:04 vm-host-colo-2 network: Shutting down interface Internal-Dev:  
[  OK  ]
Apr 20 10:51:05 vm-host-colo-2 kernel: Lab: port 1(bond0.40) entered disabled 
state
Apr 20 10:51:05 vm-host-colo-2 network: Shutting down interface Lab:  [  OK  ]
Apr 20 10:51:05 vm-host-colo-2 kernel: