On 12/10/2012 08:24 PM, Antoni Segura Puimedon wrote:
Hello everybody,

We found some unexpected behavior with bonds and we'd like to discuss it.
Please, read the forwarded messages.



----- Forwarded Message -----
From: "Dan Kenigsberg" <dan...@redhat.com>
To: "Antoni Segura Puimedon" <asegu...@redhat.com>
Cc: "Livnat Peer" <lp...@redhat.com>, "Igor Lvovsky" <ilvov...@redhat.com>
Sent: Monday, December 10, 2012 1:03:48 PM
Subject: Re: Bonding, ifcfg and luck

On Mon, Dec 10, 2012 at 06:47:58AM -0500, Antoni Segura Puimedon
Hi all,

I discussed this briefly with Livnat over the phone and mentioned
it to Dan.
The issue that we have is that, if I understand correctly our
configNetwork, it could very well be that it works by means of good
design with
a side-dish of luck.

I'll explain myself:
By design, as documented in
"All slaves of bond0 have the same MAC address (HWaddr) as bond0
for all modes
except TLB and ALB that require a unique MAC address for each

Thus, all operations on the slave interfaces after they are added
to the bond
(except on TLB and ALB modes) that rely on ifcfg will fail with a
message like:
"Device eth3 has different MAC address than expected, ignoring.",
and no
ifup/ifdown will be performed.

Currently, we were not noticing this, because we were ignoring
errors in ifdown and ifup, but http://gerrit.ovirt.org/#/c/8415/
shed light on
the matter. As you can see in the following example (bonding mode
4) the
behavior is just as documented:

     [root@rhel64 ~]# cat /sys/class/net/eth*/address
     52:54:00:ac:32:1b <-----------------
     [root@rhel64 ~]# echo "+eth2" >
     [root@rhel64 ~]# echo "+eth3" >
     [root@rhel64 ~]# cat /sys/class/net/eth*/address
     52:54:00:51:50:49 <-----------------
     [root@rhel64 ~]# echo "-eth3" >
     [root@rhel64 ~]# cat /sys/class/net/eth*/address
     52:54:00:ac:32:1b <-----------------

Obviously, this means that, for example, when we add a bridge on
top of a bond,
the ifdown, ifup of the bond slaves will be completely fruitless
luckily that doesn't prevent them from working).

Sorry, thi is not obvious to me.
When we change something in a nic, we first take it down (which break
away from the bond), change it, and then take it up again (and back
the bond).

I did not understand which flow of configuration leads us to the
"unexpected mac" error. I hope that we can circumvent it.
I get the same question. The warning message should be only seen when you run ifup on bonding or its slave, which is already up, otherwise the slave nic's mac address should hold its own permanent mac address. If the bonding is down before, you shouldn't see this message
because the nic is not enslaved.

To solve this issue on the ifcfg based operation we could either:
- Continue ignoring these issues and either not do ifup ifdown for
   slaves or catch the specific error and ignore it.
That's reasonable, for a hack.

- Modify the ifcfg files of the slaves after they are enslaved to
reflect the
   MAC addr of /sys/class/net/bond0/address. Modify the ifcfg files
   after the
   bond is destroyed to reflect their own addresses as in
I do not undestand this solution at all... Fixing initscripts to
the permanent mac address instead of the bond's one makes more sense
me. ( /proc/net/bonding/bond0 has "Permanent HW addr: " )

Livnat made me note that this behavior can be a problem to the anti
mac-spoofing rules that we add to iptables, as they rely on the
identity device
-macaddr to work and, obviously, in most bonding modes that is
broken unless
the device's macaddr is the one chosen for the bond.
Right. I suppose we can open a bug about it: in-guest bond does not
with mac-no-spoofing. I have a vague memory of discussing this with
lpeer few months back, but it somehow slipped my mind.
I am not sure why we need bonding inside guest. Bonding can provide link failover and bandwidth aggregation. We can achieve that by setting up bonding on host. If we really need it, we could workaround it by defining a netfilter for each vm, which allow the traffic from all the mac addresses belong to it. To be exact, we also could make qemu generate an event to libvirt on guest vif's mac address change, and then libvirt can validate the change and updating it's data accordingly.

Well, I think that is all for this issue. We should discuss which
is the best
approach for this before we move on with patches that account for
ifup ifdown
return information.



vdsm-devel mailing list

vdsm-devel mailing list

Reply via email to