----- Original Message -----
> From: "Alon Bar-Lev" <alo...@redhat.com>
> To: "Antoni Segura Puimedon" <asegu...@redhat.com>
> Cc: vdsm-devel@lists.fedorahosted.org
> Sent: Monday, December 10, 2012 2:07:38 PM
> Subject: Re: [vdsm] Fwd: Bonding, bridges and ifcfg
> 
> Hi,
> 
> Just to make sure... working in non-persistent mode will eliminate
> these kind of issues, right?
Yes, I'm quite sure that working directly with the kernel through netlink
or the ip tools would not exhibit the issues here mentioned.
> 
> Alon
> 
> ----- Original Message -----
> > From: "Antoni Segura Puimedon" <asegu...@redhat.com>
> > To: vdsm-devel@lists.fedorahosted.org
> > Sent: Monday, December 10, 2012 2:24:11 PM
> > Subject: [vdsm] Fwd: Bonding, bridges and ifcfg
> > 
> > Hello everybody,
> > 
> > We found some unexpected behavior with bonds and we'd like to
> > discuss
> > it.
> > Please, read the forwarded messages.
> > 
> > Best,
> > 
> > Toni
> > 
> > ----- Forwarded Message -----
> > > From: "Dan Kenigsberg" <dan...@redhat.com>
> > > To: "Antoni Segura Puimedon" <asegu...@redhat.com>
> > > Cc: "Livnat Peer" <lp...@redhat.com>, "Igor Lvovsky"
> > > <ilvov...@redhat.com>
> > > Sent: Monday, December 10, 2012 1:03:48 PM
> > > Subject: Re: Bonding, ifcfg and luck
> > > 
> > > On Mon, Dec 10, 2012 at 06:47:58AM -0500, Antoni Segura Puimedon
> > > wrote:
> > > > Hi all,
> > > > 
> > > > I discussed this briefly with Livnat over the phone and
> > > > mentioned
> > > > it to Dan.
> > > > The issue that we have is that, if I understand correctly our
> > > > current
> > > > configNetwork, it could very well be that it works by means of
> > > > good
> > > > design with
> > > > a side-dish of luck.
> > > > 
> > > > I'll explain myself:
> > > > By design, as documented in
> > > > http://www.kernel.org/doc/Documentation/networking/bonding.txt:
> > > > "All slaves of bond0 have the same MAC address (HWaddr) as
> > > > bond0
> > > > for all modes
> > > > except TLB and ALB that require a unique MAC address for each
> > > > slave."
> > > > 
> > > > Thus, all operations on the slave interfaces after they are
> > > > added
> > > > to the bond
> > > > (except on TLB and ALB modes) that rely on ifcfg will fail with
> > > > a
> > > > message like:
> > > > "Device eth3 has different MAC address than expected,
> > > > ignoring.",
> > > > and no
> > > > ifup/ifdown will be performed.
> > > > 
> > > > Currently, we were not noticing this, because we were ignoring
> > > > completely
> > > > errors in ifdown and ifup, but
> > > > http://gerrit.ovirt.org/#/c/8415/
> > > > shed light on
> > > > the matter. As you can see in the following example (bonding
> > > > mode
> > > > 4) the
> > > > behavior is just as documented:
> > > > 
> > > >     [root@rhel64 ~]# cat /sys/class/net/eth*/address
> > > >     52:54:00:a2:b4:50
> > > >     52:54:00:3f:9b:28
> > > >     52:54:00:51:50:49
> > > >     52:54:00:ac:32:1b <-----------------
> > > >     [root@rhel64 ~]# echo "+eth2" >
> > > >     /sys/class/net/bond0/bonding/slaves
> > > >     [root@rhel64 ~]# echo "+eth3" >
> > > >     /sys/class/net/bond0/bonding/slaves
> > > >     [root@rhel64 ~]# cat /sys/class/net/eth*/address
> > > >     52:54:00:a2:b4:50
> > > >     52:54:00:3f:9b:28
> > > >     52:54:00:51:50:49
> > > >     52:54:00:51:50:49 <-----------------
> > > >     [root@rhel64 ~]# echo "-eth3" >
> > > >     /sys/class/net/bond0/bonding/slaves
> > > >     [root@rhel64 ~]# cat /sys/class/net/eth*/address
> > > >     52:54:00:a2:b4:50
> > > >     52:54:00:3f:9b:28
> > > >     52:54:00:51:50:49
> > > >     52:54:00:ac:32:1b <-----------------
> > > > 
> > > > Obviously, this means that, for example, when we add a bridge
> > > > on
> > > > top of a bond,
> > > > the ifdown, ifup of the bond slaves will be completely
> > > > fruitless
> > > > (although
> > > > luckily that doesn't prevent them from working).
> > > 
> > > 
> > > Sorry, thi is not obvious to me.
> > > When we change something in a nic, we first take it down (which
> > > break
> > > it
> > > away from the bond), change it, and then take it up again (and
> > > back
> > > to
> > > the bond).
> > > 
> > > I did not understand which flow of configuration leads us to the
> > > "unexpected mac" error. I hope that we can circumvent it.
> > > 
> > > 
> > > > 
> > > > To solve this issue on the ifcfg based operation we could
> > > > either:
> > > > - Continue ignoring these issues and either not do ifup ifdown
> > > > for
> > > > bonding
> > > >   slaves or catch the specific error and ignore it.
> > > 
> > > That's reasonable, for a hack.
> > > 
> > > > - Modify the ifcfg files of the slaves after they are enslaved
> > > > to
> > > > reflect the
> > > >   MAC addr of /sys/class/net/bond0/address. Modify the ifcfg
> > > >   files
> > > >   after the
> > > >   bond is destroyed to reflect their own addresses as in
> > > >   /sys/class/net/ethx/address
> > > 
> > > I do not undestand this solution at all... Fixing initscripts to
> > > expect
> > > the permanent mac address instead of the bond's one makes more
> > > sense
> > > to
> > > me. ( /proc/net/bonding/bond0 has "Permanent HW addr: " )
> > > 
> > > > 
> > > > Livnat made me note that this behavior can be a problem to the
> > > > anti
> > > > mac-spoofing rules that we add to iptables, as they rely on the
> > > > identity device
> > > > -macaddr to work and, obviously, in most bonding modes that is
> > > > broken unless
> > > > the device's macaddr is the one chosen for the bond.
> > > 
> > > Right. I suppose we can open a bug about it: in-guest bond does
> > > not
> > > work
> > > with mac-no-spoofing. I have a vague memory of discussing this
> > > with
> > > lpeer few months back, but it somehow slipped my mind.
> > > 
> > > 
> > > > Well, I think that is all for this issue. We should discuss
> > > > which
> > > > is the best
> > > > approach for this before we move on with patches that account
> > > > for
> > > > ifup ifdown
> > > > return information.
> > > > 
> > > > Best,
> > > > 
> > > > Toni
> > > > 
> > > 
> > _______________________________________________
> > vdsm-devel mailing list
> > vdsm-devel@lists.fedorahosted.org
> > https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
> > 
> 
_______________________________________________
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel

Reply via email to