On 12/12/2012 03:13 PM, Igor Lvovsky wrote:

----- Original Message -----
From: "Antoni Segura Puimedon" <asegu...@redhat.com>
To: "Dan Kenigsberg" <dan...@redhat.com>
Cc: "Mark Wu" <wu...@linux.vnet.ibm.com>, vdsm-devel@lists.fedorahosted.org, "Igor 
Lvovsky" <ilvov...@redhat.com>
Sent: Wednesday, December 12, 2012 1:45:54 AM
Subject: Re: [vdsm] Fwd: Bonding, bridges and ifcfg



----- Original Message -----
From: "Dan Kenigsberg" <dan...@redhat.com>
To: "Antoni Segura Puimedon" <asegu...@redhat.com>
Cc: "Mark Wu" <wu...@linux.vnet.ibm.com>,
vdsm-devel@lists.fedorahosted.org, "Igor Lvovsky"
<ilvov...@redhat.com>
Sent: Tuesday, December 11, 2012 10:38:45 PM
Subject: Re: [vdsm] Fwd: Bonding, bridges and ifcfg

On Tue, Dec 11, 2012 at 04:21:51PM -0500, Antoni Segura Puimedon
wrote:

----- Original Message -----
From: "Dan Kenigsberg" <dan...@redhat.com>
To: "Antoni Segura Puimedon" <asegu...@redhat.com>, "Igor
Lvovsky" <ilvov...@redhat.com>
Cc: "Mark Wu" <wu...@linux.vnet.ibm.com>,
vdsm-devel@lists.fedorahosted.org
Sent: Tuesday, December 11, 2012 7:44:05 PM
Subject: Re: [vdsm] Fwd: Bonding, bridges and ifcfg

On Tue, Dec 11, 2012 at 11:42:49AM -0500, Antoni Segura
Puimedon
wrote:

----- Original Message -----
From: "Mark Wu" <wu...@linux.vnet.ibm.com>
To: "Antoni Segura Puimedon" <asegu...@redhat.com>
Cc: vdsm-devel@lists.fedorahosted.org
Sent: Tuesday, December 11, 2012 6:42:45 AM
Subject: Re: [vdsm] Fwd: Bonding, bridges and ifcfg

On 12/10/2012 08:24 PM, Antoni Segura Puimedon wrote:
Hello everybody,

We found some unexpected behavior with bonds and we'd
like
to
discuss it.
Please, read the forwarded messages.

Best,

Toni

----- Forwarded Message -----
From: "Dan Kenigsberg" <dan...@redhat.com>
To: "Antoni Segura Puimedon" <asegu...@redhat.com>
Cc: "Livnat Peer" <lp...@redhat.com>, "Igor Lvovsky"
<ilvov...@redhat.com>
Sent: Monday, December 10, 2012 1:03:48 PM
Subject: Re: Bonding, ifcfg and luck

On Mon, Dec 10, 2012 at 06:47:58AM -0500, Antoni Segura
Puimedon
wrote:
Hi all,

I discussed this briefly with Livnat over the phone and
mentioned
it to Dan.
The issue that we have is that, if I understand
correctly
our
current
configNetwork, it could very well be that it works by
means
of
good
design with
a side-dish of luck.

I'll explain myself:
By design, as documented in
http://www.kernel.org/doc/Documentation/networking/bonding.txt:
"All slaves of bond0 have the same MAC address (HWaddr)
as
bond0
for all modes
except TLB and ALB that require a unique MAC address
for
each
slave."

Thus, all operations on the slave interfaces after they
are
added
to the bond
(except on TLB and ALB modes) that rely on ifcfg will
fail
with a
message like:
"Device eth3 has different MAC address than expected,
ignoring.",
and no
ifup/ifdown will be performed.

Currently, we were not noticing this, because we were
ignoring
completely
errors in ifdown and ifup, but
http://gerrit.ovirt.org/#/c/8415/
shed light on
the matter. As you can see in the following example
(bonding
mode
4) the
behavior is just as documented:

      [root@rhel64 ~]# cat /sys/class/net/eth*/address
      52:54:00:a2:b4:50
      52:54:00:3f:9b:28
      52:54:00:51:50:49
      52:54:00:ac:32:1b <-----------------
      [root@rhel64 ~]# echo "+eth2" >
      /sys/class/net/bond0/bonding/slaves
      [root@rhel64 ~]# echo "+eth3" >
      /sys/class/net/bond0/bonding/slaves
      [root@rhel64 ~]# cat /sys/class/net/eth*/address
      52:54:00:a2:b4:50
      52:54:00:3f:9b:28
      52:54:00:51:50:49
      52:54:00:51:50:49 <-----------------
      [root@rhel64 ~]# echo "-eth3" >
      /sys/class/net/bond0/bonding/slaves
      [root@rhel64 ~]# cat /sys/class/net/eth*/address
      52:54:00:a2:b4:50
      52:54:00:3f:9b:28
      52:54:00:51:50:49
      52:54:00:ac:32:1b <-----------------

Obviously, this means that, for example, when we add a
bridge
on
top of a bond,
the ifdown, ifup of the bond slaves will be completely
fruitless
(although
luckily that doesn't prevent them from working).
Sorry, thi is not obvious to me.
When we change something in a nic, we first take it down
(which
break
it
away from the bond), change it, and then take it up
again
(and
back
to
the bond).

I did not understand which flow of configuration leads
us
to
the
"unexpected mac" error. I hope that we can circumvent
it.
I get the same question.  The warning message should be
only
seen
when
you run ifup on bonding or its slave, which is already up,
otherwise
the slave nic's mac address should hold its own permanent
mac
address.
If the bonding is down before, you shouldn't see this
message
because the nic is not enslaved.
It's easy enough to reproduce. To the current HEAD do:

vdsClient 0 addNetwork bridge=kaboom bonding=bond0
nics=eth1,eth2
bridged=True

Go to /var/log/vdsm/vdsm.log and you'll see that:
http://pastebin.com/09eRNzv6

On line 33 you'll see that the ifup of eth2 is FAILED.
Ok, I think this is a bug in the sequence of our ifup'ing.
Reviewing
ifup-eth I see that in its "if bonding device" stanza there's

if [ "$ISALIAS" = no ] && is_bonding_device ${DEVICE} ; then
...
     for device in $(LANG=C egrep -l
     "^[[:space:]]*MASTER=\"?${DEVICE}\"?[[:space:]]*$"
     /etc/sysconfig/network-scripts/ifcfg-*) ; do
             is_ignored_file "$device" && continue
             /sbin/ifup ${device##*/}
...

meaning that if we prepare ifcfg-eth* files on time, the
interfaces
would be
brought up by means of `ifup bond??`; and it seems that there
is
no
need
for a specific `ifup eth0` at all.
Yes this is the same conclusion I arrived when I looked at it
earlier,
Basically on bonding case we can just not do the ifups of the
underlying
devices.
s/we can/we should/ .
And it would also save us some wasted time.
Agreed!
what about ifdown? Did you look at it?
Is it redundant to ifdown NICs too if we ifdown bond?
Yes, the same as ifup:

if is_bonding_device ${DEVICE} ; then
for device in $(LANG=C egrep -l "^[[:space:]]*MASTER=\"?${DEVICE}\"?" /etc/sysconfig/network-scripts/ifcfg-*) ; do
        is_ignored_file "$device" && continue
        /sbin/ifdown ${device##*/}
    done



_______________________________________________
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel

Reply via email to