On Mon, Dec 10, 2012 at 08:07:38AM -0500, Alon Bar-Lev wrote:
> Hi,
> 
> Just to make sure... working in non-persistent mode will eliminate these kind 
> of issues, right?


No. It would eliminate the need to debug initscripts. But it would require
vdsm developer of an intimate recognition of kernel quirks.

We'd have fewer building blocks, and less of chance for incompatibility.
But we would need to reimplement (some of) the logic within ifup script.

> 
> Alon
> 
> ----- Original Message -----
> > From: "Antoni Segura Puimedon" <asegu...@redhat.com>
> > To: vdsm-devel@lists.fedorahosted.org
> > Sent: Monday, December 10, 2012 2:24:11 PM
> > Subject: [vdsm] Fwd: Bonding, bridges and ifcfg
> > 
> > Hello everybody,
> > 
> > We found some unexpected behavior with bonds and we'd like to discuss
> > it.
> > Please, read the forwarded messages.
> > 
> > Best,
> > 
> > Toni
> > 
> > ----- Forwarded Message -----
> > > From: "Dan Kenigsberg" <dan...@redhat.com>
> > > To: "Antoni Segura Puimedon" <asegu...@redhat.com>
> > > Cc: "Livnat Peer" <lp...@redhat.com>, "Igor Lvovsky"
> > > <ilvov...@redhat.com>
> > > Sent: Monday, December 10, 2012 1:03:48 PM
> > > Subject: Re: Bonding, ifcfg and luck
> > > 
> > > On Mon, Dec 10, 2012 at 06:47:58AM -0500, Antoni Segura Puimedon
> > > wrote:
> > > > Hi all,
> > > > 
> > > > I discussed this briefly with Livnat over the phone and mentioned
> > > > it to Dan.
> > > > The issue that we have is that, if I understand correctly our
> > > > current
> > > > configNetwork, it could very well be that it works by means of
> > > > good
> > > > design with
> > > > a side-dish of luck.
> > > > 
> > > > I'll explain myself:
> > > > By design, as documented in
> > > > http://www.kernel.org/doc/Documentation/networking/bonding.txt:
> > > > "All slaves of bond0 have the same MAC address (HWaddr) as bond0
> > > > for all modes
> > > > except TLB and ALB that require a unique MAC address for each
> > > > slave."
> > > > 
> > > > Thus, all operations on the slave interfaces after they are added
> > > > to the bond
> > > > (except on TLB and ALB modes) that rely on ifcfg will fail with a
> > > > message like:
> > > > "Device eth3 has different MAC address than expected, ignoring.",
> > > > and no
> > > > ifup/ifdown will be performed.
> > > > 
> > > > Currently, we were not noticing this, because we were ignoring
> > > > completely
> > > > errors in ifdown and ifup, but http://gerrit.ovirt.org/#/c/8415/
> > > > shed light on
> > > > the matter. As you can see in the following example (bonding mode
> > > > 4) the
> > > > behavior is just as documented:
> > > > 
> > > >     [root@rhel64 ~]# cat /sys/class/net/eth*/address
> > > >     52:54:00:a2:b4:50
> > > >     52:54:00:3f:9b:28
> > > >     52:54:00:51:50:49
> > > >     52:54:00:ac:32:1b <-----------------
> > > >     [root@rhel64 ~]# echo "+eth2" >
> > > >     /sys/class/net/bond0/bonding/slaves
> > > >     [root@rhel64 ~]# echo "+eth3" >
> > > >     /sys/class/net/bond0/bonding/slaves
> > > >     [root@rhel64 ~]# cat /sys/class/net/eth*/address
> > > >     52:54:00:a2:b4:50
> > > >     52:54:00:3f:9b:28
> > > >     52:54:00:51:50:49
> > > >     52:54:00:51:50:49 <-----------------
> > > >     [root@rhel64 ~]# echo "-eth3" >
> > > >     /sys/class/net/bond0/bonding/slaves
> > > >     [root@rhel64 ~]# cat /sys/class/net/eth*/address
> > > >     52:54:00:a2:b4:50
> > > >     52:54:00:3f:9b:28
> > > >     52:54:00:51:50:49
> > > >     52:54:00:ac:32:1b <-----------------
> > > > 
> > > > Obviously, this means that, for example, when we add a bridge on
> > > > top of a bond,
> > > > the ifdown, ifup of the bond slaves will be completely fruitless
> > > > (although
> > > > luckily that doesn't prevent them from working).
> > > 
> > > 
> > > Sorry, thi is not obvious to me.
> > > When we change something in a nic, we first take it down (which
> > > break
> > > it
> > > away from the bond), change it, and then take it up again (and back
> > > to
> > > the bond).
> > > 
> > > I did not understand which flow of configuration leads us to the
> > > "unexpected mac" error. I hope that we can circumvent it.
> > > 
> > > 
> > > > 
> > > > To solve this issue on the ifcfg based operation we could either:
> > > > - Continue ignoring these issues and either not do ifup ifdown
> > > > for
> > > > bonding
> > > >   slaves or catch the specific error and ignore it.
> > > 
> > > That's reasonable, for a hack.
> > > 
> > > > - Modify the ifcfg files of the slaves after they are enslaved to
> > > > reflect the
> > > >   MAC addr of /sys/class/net/bond0/address. Modify the ifcfg
> > > >   files
> > > >   after the
> > > >   bond is destroyed to reflect their own addresses as in
> > > >   /sys/class/net/ethx/address
> > > 
> > > I do not undestand this solution at all... Fixing initscripts to
> > > expect
> > > the permanent mac address instead of the bond's one makes more
> > > sense
> > > to
> > > me. ( /proc/net/bonding/bond0 has "Permanent HW addr: " )
> > > 
> > > > 
> > > > Livnat made me note that this behavior can be a problem to the
> > > > anti
> > > > mac-spoofing rules that we add to iptables, as they rely on the
> > > > identity device
> > > > -macaddr to work and, obviously, in most bonding modes that is
> > > > broken unless
> > > > the device's macaddr is the one chosen for the bond.
> > > 
> > > Right. I suppose we can open a bug about it: in-guest bond does not
> > > work
> > > with mac-no-spoofing. I have a vague memory of discussing this with
> > > lpeer few months back, but it somehow slipped my mind.
> > > 
> > > 
> > > > Well, I think that is all for this issue. We should discuss which
> > > > is the best
> > > > approach for this before we move on with patches that account for
> > > > ifup ifdown
> > > > return information.
> > > > 
> > > > Best,
> > > > 
> > > > Toni
> > > > 
> > > 
> > _______________________________________________
> > vdsm-devel mailing list
> > vdsm-devel@lists.fedorahosted.org
> > https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
> > 
> _______________________________________________
> vdsm-devel mailing list
> vdsm-devel@lists.fedorahosted.org
> https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
_______________________________________________
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel

Reply via email to