Hi,

Just to make sure... working in non-persistent mode will eliminate these kind 
of issues, right?

Alon

----- Original Message -----
> From: "Antoni Segura Puimedon" <asegu...@redhat.com>
> To: vdsm-devel@lists.fedorahosted.org
> Sent: Monday, December 10, 2012 2:24:11 PM
> Subject: [vdsm] Fwd: Bonding, bridges and ifcfg
> 
> Hello everybody,
> 
> We found some unexpected behavior with bonds and we'd like to discuss
> it.
> Please, read the forwarded messages.
> 
> Best,
> 
> Toni
> 
> ----- Forwarded Message -----
> > From: "Dan Kenigsberg" <dan...@redhat.com>
> > To: "Antoni Segura Puimedon" <asegu...@redhat.com>
> > Cc: "Livnat Peer" <lp...@redhat.com>, "Igor Lvovsky"
> > <ilvov...@redhat.com>
> > Sent: Monday, December 10, 2012 1:03:48 PM
> > Subject: Re: Bonding, ifcfg and luck
> > 
> > On Mon, Dec 10, 2012 at 06:47:58AM -0500, Antoni Segura Puimedon
> > wrote:
> > > Hi all,
> > > 
> > > I discussed this briefly with Livnat over the phone and mentioned
> > > it to Dan.
> > > The issue that we have is that, if I understand correctly our
> > > current
> > > configNetwork, it could very well be that it works by means of
> > > good
> > > design with
> > > a side-dish of luck.
> > > 
> > > I'll explain myself:
> > > By design, as documented in
> > > http://www.kernel.org/doc/Documentation/networking/bonding.txt:
> > > "All slaves of bond0 have the same MAC address (HWaddr) as bond0
> > > for all modes
> > > except TLB and ALB that require a unique MAC address for each
> > > slave."
> > > 
> > > Thus, all operations on the slave interfaces after they are added
> > > to the bond
> > > (except on TLB and ALB modes) that rely on ifcfg will fail with a
> > > message like:
> > > "Device eth3 has different MAC address than expected, ignoring.",
> > > and no
> > > ifup/ifdown will be performed.
> > > 
> > > Currently, we were not noticing this, because we were ignoring
> > > completely
> > > errors in ifdown and ifup, but http://gerrit.ovirt.org/#/c/8415/
> > > shed light on
> > > the matter. As you can see in the following example (bonding mode
> > > 4) the
> > > behavior is just as documented:
> > > 
> > >     [root@rhel64 ~]# cat /sys/class/net/eth*/address
> > >     52:54:00:a2:b4:50
> > >     52:54:00:3f:9b:28
> > >     52:54:00:51:50:49
> > >     52:54:00:ac:32:1b <-----------------
> > >     [root@rhel64 ~]# echo "+eth2" >
> > >     /sys/class/net/bond0/bonding/slaves
> > >     [root@rhel64 ~]# echo "+eth3" >
> > >     /sys/class/net/bond0/bonding/slaves
> > >     [root@rhel64 ~]# cat /sys/class/net/eth*/address
> > >     52:54:00:a2:b4:50
> > >     52:54:00:3f:9b:28
> > >     52:54:00:51:50:49
> > >     52:54:00:51:50:49 <-----------------
> > >     [root@rhel64 ~]# echo "-eth3" >
> > >     /sys/class/net/bond0/bonding/slaves
> > >     [root@rhel64 ~]# cat /sys/class/net/eth*/address
> > >     52:54:00:a2:b4:50
> > >     52:54:00:3f:9b:28
> > >     52:54:00:51:50:49
> > >     52:54:00:ac:32:1b <-----------------
> > > 
> > > Obviously, this means that, for example, when we add a bridge on
> > > top of a bond,
> > > the ifdown, ifup of the bond slaves will be completely fruitless
> > > (although
> > > luckily that doesn't prevent them from working).
> > 
> > 
> > Sorry, thi is not obvious to me.
> > When we change something in a nic, we first take it down (which
> > break
> > it
> > away from the bond), change it, and then take it up again (and back
> > to
> > the bond).
> > 
> > I did not understand which flow of configuration leads us to the
> > "unexpected mac" error. I hope that we can circumvent it.
> > 
> > 
> > > 
> > > To solve this issue on the ifcfg based operation we could either:
> > > - Continue ignoring these issues and either not do ifup ifdown
> > > for
> > > bonding
> > >   slaves or catch the specific error and ignore it.
> > 
> > That's reasonable, for a hack.
> > 
> > > - Modify the ifcfg files of the slaves after they are enslaved to
> > > reflect the
> > >   MAC addr of /sys/class/net/bond0/address. Modify the ifcfg
> > >   files
> > >   after the
> > >   bond is destroyed to reflect their own addresses as in
> > >   /sys/class/net/ethx/address
> > 
> > I do not undestand this solution at all... Fixing initscripts to
> > expect
> > the permanent mac address instead of the bond's one makes more
> > sense
> > to
> > me. ( /proc/net/bonding/bond0 has "Permanent HW addr: " )
> > 
> > > 
> > > Livnat made me note that this behavior can be a problem to the
> > > anti
> > > mac-spoofing rules that we add to iptables, as they rely on the
> > > identity device
> > > -macaddr to work and, obviously, in most bonding modes that is
> > > broken unless
> > > the device's macaddr is the one chosen for the bond.
> > 
> > Right. I suppose we can open a bug about it: in-guest bond does not
> > work
> > with mac-no-spoofing. I have a vague memory of discussing this with
> > lpeer few months back, but it somehow slipped my mind.
> > 
> > 
> > > Well, I think that is all for this issue. We should discuss which
> > > is the best
> > > approach for this before we move on with patches that account for
> > > ifup ifdown
> > > return information.
> > > 
> > > Best,
> > > 
> > > Toni
> > > 
> > 
> _______________________________________________
> vdsm-devel mailing list
> vdsm-devel@lists.fedorahosted.org
> https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
> 
_______________________________________________
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel

Reply via email to