Hello everybody, We found some unexpected behavior with bonds and we'd like to discuss it. Please, read the forwarded messages.
Best, Toni ----- Forwarded Message ----- > From: "Dan Kenigsberg" <dan...@redhat.com> > To: "Antoni Segura Puimedon" <asegu...@redhat.com> > Cc: "Livnat Peer" <lp...@redhat.com>, "Igor Lvovsky" <ilvov...@redhat.com> > Sent: Monday, December 10, 2012 1:03:48 PM > Subject: Re: Bonding, ifcfg and luck > > On Mon, Dec 10, 2012 at 06:47:58AM -0500, Antoni Segura Puimedon > wrote: > > Hi all, > > > > I discussed this briefly with Livnat over the phone and mentioned > > it to Dan. > > The issue that we have is that, if I understand correctly our > > current > > configNetwork, it could very well be that it works by means of good > > design with > > a side-dish of luck. > > > > I'll explain myself: > > By design, as documented in > > http://www.kernel.org/doc/Documentation/networking/bonding.txt: > > "All slaves of bond0 have the same MAC address (HWaddr) as bond0 > > for all modes > > except TLB and ALB that require a unique MAC address for each > > slave." > > > > Thus, all operations on the slave interfaces after they are added > > to the bond > > (except on TLB and ALB modes) that rely on ifcfg will fail with a > > message like: > > "Device eth3 has different MAC address than expected, ignoring.", > > and no > > ifup/ifdown will be performed. > > > > Currently, we were not noticing this, because we were ignoring > > completely > > errors in ifdown and ifup, but http://gerrit.ovirt.org/#/c/8415/ > > shed light on > > the matter. As you can see in the following example (bonding mode > > 4) the > > behavior is just as documented: > > > > [root@rhel64 ~]# cat /sys/class/net/eth*/address > > 52:54:00:a2:b4:50 > > 52:54:00:3f:9b:28 > > 52:54:00:51:50:49 > > 52:54:00:ac:32:1b <----------------- > > [root@rhel64 ~]# echo "+eth2" > > > /sys/class/net/bond0/bonding/slaves > > [root@rhel64 ~]# echo "+eth3" > > > /sys/class/net/bond0/bonding/slaves > > [root@rhel64 ~]# cat /sys/class/net/eth*/address > > 52:54:00:a2:b4:50 > > 52:54:00:3f:9b:28 > > 52:54:00:51:50:49 > > 52:54:00:51:50:49 <----------------- > > [root@rhel64 ~]# echo "-eth3" > > > /sys/class/net/bond0/bonding/slaves > > [root@rhel64 ~]# cat /sys/class/net/eth*/address > > 52:54:00:a2:b4:50 > > 52:54:00:3f:9b:28 > > 52:54:00:51:50:49 > > 52:54:00:ac:32:1b <----------------- > > > > Obviously, this means that, for example, when we add a bridge on > > top of a bond, > > the ifdown, ifup of the bond slaves will be completely fruitless > > (although > > luckily that doesn't prevent them from working). > > > Sorry, thi is not obvious to me. > When we change something in a nic, we first take it down (which break > it > away from the bond), change it, and then take it up again (and back > to > the bond). > > I did not understand which flow of configuration leads us to the > "unexpected mac" error. I hope that we can circumvent it. > > > > > > To solve this issue on the ifcfg based operation we could either: > > - Continue ignoring these issues and either not do ifup ifdown for > > bonding > > slaves or catch the specific error and ignore it. > > That's reasonable, for a hack. > > > - Modify the ifcfg files of the slaves after they are enslaved to > > reflect the > > MAC addr of /sys/class/net/bond0/address. Modify the ifcfg files > > after the > > bond is destroyed to reflect their own addresses as in > > /sys/class/net/ethx/address > > I do not undestand this solution at all... Fixing initscripts to > expect > the permanent mac address instead of the bond's one makes more sense > to > me. ( /proc/net/bonding/bond0 has "Permanent HW addr: " ) > > > > > Livnat made me note that this behavior can be a problem to the anti > > mac-spoofing rules that we add to iptables, as they rely on the > > identity device > > -macaddr to work and, obviously, in most bonding modes that is > > broken unless > > the device's macaddr is the one chosen for the bond. > > Right. I suppose we can open a bug about it: in-guest bond does not > work > with mac-no-spoofing. I have a vague memory of discussing this with > lpeer few months back, but it somehow slipped my mind. > > > > Well, I think that is all for this issue. We should discuss which > > is the best > > approach for this before we move on with patches that account for > > ifup ifdown > > return information. > > > > Best, > > > > Toni > > > _______________________________________________ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
