----- Original Message -----
> From: "Antoni Segura Puimedon" <asegu...@redhat.com>
> To: "Mark Wu" <wu...@linux.vnet.ibm.com>
> Cc: vdsm-devel@lists.fedorahosted.org, "Dan Kenigsberg" <dan...@redhat.com>, 
> "Igor Lvovsky" <ilvov...@redhat.com>
> Sent: Wednesday, December 12, 2012 2:18:07 PM
> Subject: Re: [vdsm] Fwd: Bonding, bridges and ifcfg
> 
> 
> 
> ----- Original Message -----
> > From: "Mark Wu" <wu...@linux.vnet.ibm.com>
> > To: "Igor Lvovsky" <ilvov...@redhat.com>
> > Cc: "Antoni Segura Puimedon" <asegu...@redhat.com>,
> > vdsm-devel@lists.fedorahosted.org, "Dan Kenigsberg"
> > <dan...@redhat.com>
> > Sent: Wednesday, December 12, 2012 8:22:29 AM
> > Subject: Re: [vdsm] Fwd: Bonding, bridges and ifcfg
> > 
> > On 12/12/2012 03:13 PM, Igor Lvovsky wrote:
> > >
> > > ----- Original Message -----
> > >> From: "Antoni Segura Puimedon" <asegu...@redhat.com>
> > >> To: "Dan Kenigsberg" <dan...@redhat.com>
> > >> Cc: "Mark Wu" <wu...@linux.vnet.ibm.com>,
> > >> vdsm-devel@lists.fedorahosted.org, "Igor Lvovsky"
> > >> <ilvov...@redhat.com>
> > >> Sent: Wednesday, December 12, 2012 1:45:54 AM
> > >> Subject: Re: [vdsm] Fwd: Bonding, bridges and ifcfg
> > >>
> > >>
> > >>
> > >> ----- Original Message -----
> > >>> From: "Dan Kenigsberg" <dan...@redhat.com>
> > >>> To: "Antoni Segura Puimedon" <asegu...@redhat.com>
> > >>> Cc: "Mark Wu" <wu...@linux.vnet.ibm.com>,
> > >>> vdsm-devel@lists.fedorahosted.org, "Igor Lvovsky"
> > >>> <ilvov...@redhat.com>
> > >>> Sent: Tuesday, December 11, 2012 10:38:45 PM
> > >>> Subject: Re: [vdsm] Fwd: Bonding, bridges and ifcfg
> > >>>
> > >>> On Tue, Dec 11, 2012 at 04:21:51PM -0500, Antoni Segura
> > >>> Puimedon
> > >>> wrote:
> > >>>>
> > >>>> ----- Original Message -----
> > >>>>> From: "Dan Kenigsberg" <dan...@redhat.com>
> > >>>>> To: "Antoni Segura Puimedon" <asegu...@redhat.com>, "Igor
> > >>>>> Lvovsky" <ilvov...@redhat.com>
> > >>>>> Cc: "Mark Wu" <wu...@linux.vnet.ibm.com>,
> > >>>>> vdsm-devel@lists.fedorahosted.org
> > >>>>> Sent: Tuesday, December 11, 2012 7:44:05 PM
> > >>>>> Subject: Re: [vdsm] Fwd: Bonding, bridges and ifcfg
> > >>>>>
> > >>>>> On Tue, Dec 11, 2012 at 11:42:49AM -0500, Antoni Segura
> > >>>>> Puimedon
> > >>>>> wrote:
> > >>>>>>
> > >>>>>> ----- Original Message -----
> > >>>>>>> From: "Mark Wu" <wu...@linux.vnet.ibm.com>
> > >>>>>>> To: "Antoni Segura Puimedon" <asegu...@redhat.com>
> > >>>>>>> Cc: vdsm-devel@lists.fedorahosted.org
> > >>>>>>> Sent: Tuesday, December 11, 2012 6:42:45 AM
> > >>>>>>> Subject: Re: [vdsm] Fwd: Bonding, bridges and ifcfg
> > >>>>>>>
> > >>>>>>> On 12/10/2012 08:24 PM, Antoni Segura Puimedon wrote:
> > >>>>>>>> Hello everybody,
> > >>>>>>>>
> > >>>>>>>> We found some unexpected behavior with bonds and we'd
> > >>>>>>>> like
> > >>>>>>>> to
> > >>>>>>>> discuss it.
> > >>>>>>>> Please, read the forwarded messages.
> > >>>>>>>>
> > >>>>>>>> Best,
> > >>>>>>>>
> > >>>>>>>> Toni
> > >>>>>>>>
> > >>>>>>>> ----- Forwarded Message -----
> > >>>>>>>>> From: "Dan Kenigsberg" <dan...@redhat.com>
> > >>>>>>>>> To: "Antoni Segura Puimedon" <asegu...@redhat.com>
> > >>>>>>>>> Cc: "Livnat Peer" <lp...@redhat.com>, "Igor Lvovsky"
> > >>>>>>>>> <ilvov...@redhat.com>
> > >>>>>>>>> Sent: Monday, December 10, 2012 1:03:48 PM
> > >>>>>>>>> Subject: Re: Bonding, ifcfg and luck
> > >>>>>>>>>
> > >>>>>>>>> On Mon, Dec 10, 2012 at 06:47:58AM -0500, Antoni Segura
> > >>>>>>>>> Puimedon
> > >>>>>>>>> wrote:
> > >>>>>>>>>> Hi all,
> > >>>>>>>>>>
> > >>>>>>>>>> I discussed this briefly with Livnat over the phone and
> > >>>>>>>>>> mentioned
> > >>>>>>>>>> it to Dan.
> > >>>>>>>>>> The issue that we have is that, if I understand
> > >>>>>>>>>> correctly
> > >>>>>>>>>> our
> > >>>>>>>>>> current
> > >>>>>>>>>> configNetwork, it could very well be that it works by
> > >>>>>>>>>> means
> > >>>>>>>>>> of
> > >>>>>>>>>> good
> > >>>>>>>>>> design with
> > >>>>>>>>>> a side-dish of luck.
> > >>>>>>>>>>
> > >>>>>>>>>> I'll explain myself:
> > >>>>>>>>>> By design, as documented in
> > >>>>>>>>>> http://www.kernel.org/doc/Documentation/networking/bonding.txt:
> > >>>>>>>>>> "All slaves of bond0 have the same MAC address (HWaddr)
> > >>>>>>>>>> as
> > >>>>>>>>>> bond0
> > >>>>>>>>>> for all modes
> > >>>>>>>>>> except TLB and ALB that require a unique MAC address
> > >>>>>>>>>> for
> > >>>>>>>>>> each
> > >>>>>>>>>> slave."
> > >>>>>>>>>>
> > >>>>>>>>>> Thus, all operations on the slave interfaces after they
> > >>>>>>>>>> are
> > >>>>>>>>>> added
> > >>>>>>>>>> to the bond
> > >>>>>>>>>> (except on TLB and ALB modes) that rely on ifcfg will
> > >>>>>>>>>> fail
> > >>>>>>>>>> with a
> > >>>>>>>>>> message like:
> > >>>>>>>>>> "Device eth3 has different MAC address than expected,
> > >>>>>>>>>> ignoring.",
> > >>>>>>>>>> and no
> > >>>>>>>>>> ifup/ifdown will be performed.
> > >>>>>>>>>>
> > >>>>>>>>>> Currently, we were not noticing this, because we were
> > >>>>>>>>>> ignoring
> > >>>>>>>>>> completely
> > >>>>>>>>>> errors in ifdown and ifup, but
> > >>>>>>>>>> http://gerrit.ovirt.org/#/c/8415/
> > >>>>>>>>>> shed light on
> > >>>>>>>>>> the matter. As you can see in the following example
> > >>>>>>>>>> (bonding
> > >>>>>>>>>> mode
> > >>>>>>>>>> 4) the
> > >>>>>>>>>> behavior is just as documented:
> > >>>>>>>>>>
> > >>>>>>>>>>       [root@rhel64 ~]# cat /sys/class/net/eth*/address
> > >>>>>>>>>>       52:54:00:a2:b4:50
> > >>>>>>>>>>       52:54:00:3f:9b:28
> > >>>>>>>>>>       52:54:00:51:50:49
> > >>>>>>>>>>       52:54:00:ac:32:1b <-----------------
> > >>>>>>>>>>       [root@rhel64 ~]# echo "+eth2" >
> > >>>>>>>>>>       /sys/class/net/bond0/bonding/slaves
> > >>>>>>>>>>       [root@rhel64 ~]# echo "+eth3" >
> > >>>>>>>>>>       /sys/class/net/bond0/bonding/slaves
> > >>>>>>>>>>       [root@rhel64 ~]# cat /sys/class/net/eth*/address
> > >>>>>>>>>>       52:54:00:a2:b4:50
> > >>>>>>>>>>       52:54:00:3f:9b:28
> > >>>>>>>>>>       52:54:00:51:50:49
> > >>>>>>>>>>       52:54:00:51:50:49 <-----------------
> > >>>>>>>>>>       [root@rhel64 ~]# echo "-eth3" >
> > >>>>>>>>>>       /sys/class/net/bond0/bonding/slaves
> > >>>>>>>>>>       [root@rhel64 ~]# cat /sys/class/net/eth*/address
> > >>>>>>>>>>       52:54:00:a2:b4:50
> > >>>>>>>>>>       52:54:00:3f:9b:28
> > >>>>>>>>>>       52:54:00:51:50:49
> > >>>>>>>>>>       52:54:00:ac:32:1b <-----------------
> > >>>>>>>>>>
> > >>>>>>>>>> Obviously, this means that, for example, when we add a
> > >>>>>>>>>> bridge
> > >>>>>>>>>> on
> > >>>>>>>>>> top of a bond,
> > >>>>>>>>>> the ifdown, ifup of the bond slaves will be completely
> > >>>>>>>>>> fruitless
> > >>>>>>>>>> (although
> > >>>>>>>>>> luckily that doesn't prevent them from working).
> > >>>>>>>>> Sorry, thi is not obvious to me.
> > >>>>>>>>> When we change something in a nic, we first take it down
> > >>>>>>>>> (which
> > >>>>>>>>> break
> > >>>>>>>>> it
> > >>>>>>>>> away from the bond), change it, and then take it up
> > >>>>>>>>> again
> > >>>>>>>>> (and
> > >>>>>>>>> back
> > >>>>>>>>> to
> > >>>>>>>>> the bond).
> > >>>>>>>>>
> > >>>>>>>>> I did not understand which flow of configuration leads
> > >>>>>>>>> us
> > >>>>>>>>> to
> > >>>>>>>>> the
> > >>>>>>>>> "unexpected mac" error. I hope that we can circumvent
> > >>>>>>>>> it.
> > >>>>>>> I get the same question.  The warning message should be
> > >>>>>>> only
> > >>>>>>> seen
> > >>>>>>> when
> > >>>>>>> you run ifup on bonding or its slave, which is already up,
> > >>>>>>> otherwise
> > >>>>>>> the slave nic's mac address should hold its own permanent
> > >>>>>>> mac
> > >>>>>>> address.
> > >>>>>>> If the bonding is down before, you shouldn't see this
> > >>>>>>> message
> > >>>>>>> because the nic is not enslaved.
> > >>>>>> It's easy enough to reproduce. To the current HEAD do:
> > >>>>>>
> > >>>>>> vdsClient 0 addNetwork bridge=kaboom bonding=bond0
> > >>>>>> nics=eth1,eth2
> > >>>>>> bridged=True
> > >>>>>>
> > >>>>>> Go to /var/log/vdsm/vdsm.log and you'll see that:
> > >>>>>> http://pastebin.com/09eRNzv6
> > >>>>>>
> > >>>>>> On line 33 you'll see that the ifup of eth2 is FAILED.
> > >>>>> Ok, I think this is a bug in the sequence of our ifup'ing.
> > >>>>> Reviewing
> > >>>>> ifup-eth I see that in its "if bonding device" stanza there's
> > >>>>>
> > >>>>> if [ "$ISALIAS" = no ] && is_bonding_device ${DEVICE} ; then
> > >>>>> ...
> > >>>>>      for device in $(LANG=C egrep -l
> > >>>>>      "^[[:space:]]*MASTER=\"?${DEVICE}\"?[[:space:]]*$"
> > >>>>>      /etc/sysconfig/network-scripts/ifcfg-*) ; do
> > >>>>>              is_ignored_file "$device" && continue
> > >>>>>              /sbin/ifup ${device##*/}
> > >>>>> ...
> > >>>>>
> > >>>>> meaning that if we prepare ifcfg-eth* files on time, the
> > >>>>> interfaces
> > >>>>> would be
> > >>>>> brought up by means of `ifup bond??`; and it seems that there
> > >>>>> is
> > >>>>> no
> > >>>>> need
> > >>>>> for a specific `ifup eth0` at all.
> > >>>> Yes this is the same conclusion I arrived when I looked at it
> > >>>> earlier,
> > >>>> Basically on bonding case we can just not do the ifups of the
> > >>>> underlying
> > >>>> devices.
> > >>> s/we can/we should/ .
> > >>> And it would also save us some wasted time.
> > >> Agreed!
> > > what about ifdown? Did you look at it?
> > > Is it redundant to ifdown NICs too if we ifdown bond?
> > Yes, the same as ifup:
> > 
> > if is_bonding_device ${DEVICE} ; then
> >      for device in $(LANG=C egrep -l
> > "^[[:space:]]*MASTER=\"?${DEVICE}\"?"
> > /etc/sysconfig/network-scripts/ifcfg-*) ; do
> >          is_ignored_file "$device" && continue
> >          /sbin/ifdown ${device##*/}
> >      done
> > 
> > >
> > 
> > 
> 
> I wonder if in F18 we are going to get bitten by NetworkManager with
> the
> mac address change. I will put on my TODO list to check that it
> doesn't
> regain control over the interface and if, submit a bug for it.
> 
when I ran tests on F18 (with NM) it includes bond tests too and it passed.
So, it looks like F18 behaves similar to RHEL with bonds.
But, you are right better to recheck it
_______________________________________________
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel

Reply via email to