----- Original Message -----
> From: "Dan Kenigsberg" <dan...@redhat.com>
> To: "Antoni Segura Puimedon" <asegu...@redhat.com>
> Cc: "Mark Wu" <wu...@linux.vnet.ibm.com>, vdsm-devel@lists.fedorahosted.org, 
> "Igor Lvovsky" <ilvov...@redhat.com>
> Sent: Tuesday, December 11, 2012 10:38:45 PM
> Subject: Re: [vdsm] Fwd: Bonding, bridges and ifcfg
> 
> On Tue, Dec 11, 2012 at 04:21:51PM -0500, Antoni Segura Puimedon
> wrote:
> > 
> > 
> > ----- Original Message -----
> > > From: "Dan Kenigsberg" <dan...@redhat.com>
> > > To: "Antoni Segura Puimedon" <asegu...@redhat.com>, "Igor
> > > Lvovsky" <ilvov...@redhat.com>
> > > Cc: "Mark Wu" <wu...@linux.vnet.ibm.com>,
> > > vdsm-devel@lists.fedorahosted.org
> > > Sent: Tuesday, December 11, 2012 7:44:05 PM
> > > Subject: Re: [vdsm] Fwd: Bonding, bridges and ifcfg
> > > 
> > > On Tue, Dec 11, 2012 at 11:42:49AM -0500, Antoni Segura Puimedon
> > > wrote:
> > > > 
> > > > 
> > > > ----- Original Message -----
> > > > > From: "Mark Wu" <wu...@linux.vnet.ibm.com>
> > > > > To: "Antoni Segura Puimedon" <asegu...@redhat.com>
> > > > > Cc: vdsm-devel@lists.fedorahosted.org
> > > > > Sent: Tuesday, December 11, 2012 6:42:45 AM
> > > > > Subject: Re: [vdsm] Fwd: Bonding, bridges and ifcfg
> > > > > 
> > > > > On 12/10/2012 08:24 PM, Antoni Segura Puimedon wrote:
> > > > > > Hello everybody,
> > > > > >
> > > > > > We found some unexpected behavior with bonds and we'd like
> > > > > > to
> > > > > > discuss it.
> > > > > > Please, read the forwarded messages.
> > > > > >
> > > > > > Best,
> > > > > >
> > > > > > Toni
> > > > > >
> > > > > > ----- Forwarded Message -----
> > > > > >> From: "Dan Kenigsberg" <dan...@redhat.com>
> > > > > >> To: "Antoni Segura Puimedon" <asegu...@redhat.com>
> > > > > >> Cc: "Livnat Peer" <lp...@redhat.com>, "Igor Lvovsky"
> > > > > >> <ilvov...@redhat.com>
> > > > > >> Sent: Monday, December 10, 2012 1:03:48 PM
> > > > > >> Subject: Re: Bonding, ifcfg and luck
> > > > > >>
> > > > > >> On Mon, Dec 10, 2012 at 06:47:58AM -0500, Antoni Segura
> > > > > >> Puimedon
> > > > > >> wrote:
> > > > > >>> Hi all,
> > > > > >>>
> > > > > >>> I discussed this briefly with Livnat over the phone and
> > > > > >>> mentioned
> > > > > >>> it to Dan.
> > > > > >>> The issue that we have is that, if I understand correctly
> > > > > >>> our
> > > > > >>> current
> > > > > >>> configNetwork, it could very well be that it works by
> > > > > >>> means
> > > > > >>> of
> > > > > >>> good
> > > > > >>> design with
> > > > > >>> a side-dish of luck.
> > > > > >>>
> > > > > >>> I'll explain myself:
> > > > > >>> By design, as documented in
> > > > > >>> http://www.kernel.org/doc/Documentation/networking/bonding.txt:
> > > > > >>> "All slaves of bond0 have the same MAC address (HWaddr)
> > > > > >>> as
> > > > > >>> bond0
> > > > > >>> for all modes
> > > > > >>> except TLB and ALB that require a unique MAC address for
> > > > > >>> each
> > > > > >>> slave."
> > > > > >>>
> > > > > >>> Thus, all operations on the slave interfaces after they
> > > > > >>> are
> > > > > >>> added
> > > > > >>> to the bond
> > > > > >>> (except on TLB and ALB modes) that rely on ifcfg will
> > > > > >>> fail
> > > > > >>> with a
> > > > > >>> message like:
> > > > > >>> "Device eth3 has different MAC address than expected,
> > > > > >>> ignoring.",
> > > > > >>> and no
> > > > > >>> ifup/ifdown will be performed.
> > > > > >>>
> > > > > >>> Currently, we were not noticing this, because we were
> > > > > >>> ignoring
> > > > > >>> completely
> > > > > >>> errors in ifdown and ifup, but
> > > > > >>> http://gerrit.ovirt.org/#/c/8415/
> > > > > >>> shed light on
> > > > > >>> the matter. As you can see in the following example
> > > > > >>> (bonding
> > > > > >>> mode
> > > > > >>> 4) the
> > > > > >>> behavior is just as documented:
> > > > > >>>
> > > > > >>>      [root@rhel64 ~]# cat /sys/class/net/eth*/address
> > > > > >>>      52:54:00:a2:b4:50
> > > > > >>>      52:54:00:3f:9b:28
> > > > > >>>      52:54:00:51:50:49
> > > > > >>>      52:54:00:ac:32:1b <-----------------
> > > > > >>>      [root@rhel64 ~]# echo "+eth2" >
> > > > > >>>      /sys/class/net/bond0/bonding/slaves
> > > > > >>>      [root@rhel64 ~]# echo "+eth3" >
> > > > > >>>      /sys/class/net/bond0/bonding/slaves
> > > > > >>>      [root@rhel64 ~]# cat /sys/class/net/eth*/address
> > > > > >>>      52:54:00:a2:b4:50
> > > > > >>>      52:54:00:3f:9b:28
> > > > > >>>      52:54:00:51:50:49
> > > > > >>>      52:54:00:51:50:49 <-----------------
> > > > > >>>      [root@rhel64 ~]# echo "-eth3" >
> > > > > >>>      /sys/class/net/bond0/bonding/slaves
> > > > > >>>      [root@rhel64 ~]# cat /sys/class/net/eth*/address
> > > > > >>>      52:54:00:a2:b4:50
> > > > > >>>      52:54:00:3f:9b:28
> > > > > >>>      52:54:00:51:50:49
> > > > > >>>      52:54:00:ac:32:1b <-----------------
> > > > > >>>
> > > > > >>> Obviously, this means that, for example, when we add a
> > > > > >>> bridge
> > > > > >>> on
> > > > > >>> top of a bond,
> > > > > >>> the ifdown, ifup of the bond slaves will be completely
> > > > > >>> fruitless
> > > > > >>> (although
> > > > > >>> luckily that doesn't prevent them from working).
> > > > > >>
> > > > > >> Sorry, thi is not obvious to me.
> > > > > >> When we change something in a nic, we first take it down
> > > > > >> (which
> > > > > >> break
> > > > > >> it
> > > > > >> away from the bond), change it, and then take it up again
> > > > > >> (and
> > > > > >> back
> > > > > >> to
> > > > > >> the bond).
> > > > > >>
> > > > > >> I did not understand which flow of configuration leads us
> > > > > >> to
> > > > > >> the
> > > > > >> "unexpected mac" error. I hope that we can circumvent it.
> > > > > I get the same question.  The warning message should be only
> > > > > seen
> > > > > when
> > > > > you run ifup on bonding or its slave, which is already up,
> > > > > otherwise
> > > > > the slave nic's mac address should hold its own permanent mac
> > > > > address.
> > > > > If the bonding is down before, you shouldn't see this message
> > > > > because the nic is not enslaved.
> > > > 
> > > > It's easy enough to reproduce. To the current HEAD do:
> > > > 
> > > > vdsClient 0 addNetwork bridge=kaboom bonding=bond0
> > > > nics=eth1,eth2
> > > > bridged=True
> > > > 
> > > > Go to /var/log/vdsm/vdsm.log and you'll see that:
> > > > http://pastebin.com/09eRNzv6
> > > > 
> > > > On line 33 you'll see that the ifup of eth2 is FAILED.
> > > 
> > > Ok, I think this is a bug in the sequence of our ifup'ing.
> > > Reviewing
> > > ifup-eth I see that in its "if bonding device" stanza there's
> > > 
> > > if [ "$ISALIAS" = no ] && is_bonding_device ${DEVICE} ; then
> > > ...
> > >     for device in $(LANG=C egrep -l
> > >     "^[[:space:]]*MASTER=\"?${DEVICE}\"?[[:space:]]*$"
> > >     /etc/sysconfig/network-scripts/ifcfg-*) ; do
> > >             is_ignored_file "$device" && continue
> > >             /sbin/ifup ${device##*/}
> > > ...
> > > 
> > > meaning that if we prepare ifcfg-eth* files on time, the
> > > interfaces
> > > would be
> > > brought up by means of `ifup bond??`; and it seems that there is
> > > no
> > > need
> > > for a specific `ifup eth0` at all.
> > 
> > Yes this is the same conclusion I arrived when I looked at it
> > earlier,
> > Basically on bonding case we can just not do the ifups of the
> > underlying
> > devices.
> 
> s/we can/we should/ .
> And it would also save us some wasted time.
Agreed!
> 
_______________________________________________
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel

Reply via email to