----- Original Message ----- > From: "Dan Kenigsberg" <dan...@redhat.com> > To: "Alon Bar-Lev" <alo...@redhat.com> > Cc: "Antoni Segura Puimedon" <asegu...@redhat.com>, > vdsm-devel@lists.fedorahosted.org > Sent: Monday, December 10, 2012 3:16:21 PM > Subject: Re: [vdsm] Fwd: Bonding, bridges and ifcfg > > On Mon, Dec 10, 2012 at 08:07:38AM -0500, Alon Bar-Lev wrote: > > Hi, > > > > Just to make sure... working in non-persistent mode will eliminate > > these kind of issues, right? > > > No. It would eliminate the need to debug initscripts. But it would > require > vdsm developer of an intimate recognition of kernel quirks. > > We'd have fewer building blocks, and less of chance for > incompatibility. > But we would need to reimplement (some of) the logic within ifup > script.
Sure you need to reimplement ifup and ifdown functionality as you would not use these... You will not have fewer building blocks if you will break the fedora/redhat border, actually if you go non persistent you will have fewer of these and be more portable as you have one kernel (linux) to support. vdsm developer [should] already require intimate recognition of the kernel, see bellow one example. It is just that even if one has intimate recognition of the kernel, working via primitive tools like rhel/fedora network-script only make it harder to produce the desired outcome, while having full control over the process and the result. > > > > > Alon > > > > ----- Original Message ----- > > > From: "Antoni Segura Puimedon" <asegu...@redhat.com> > > > To: vdsm-devel@lists.fedorahosted.org > > > Sent: Monday, December 10, 2012 2:24:11 PM > > > Subject: [vdsm] Fwd: Bonding, bridges and ifcfg > > > > > > Hello everybody, > > > > > > We found some unexpected behavior with bonds and we'd like to > > > discuss > > > it. > > > Please, read the forwarded messages. > > > > > > Best, > > > > > > Toni > > > > > > ----- Forwarded Message ----- > > > > From: "Dan Kenigsberg" <dan...@redhat.com> > > > > To: "Antoni Segura Puimedon" <asegu...@redhat.com> > > > > Cc: "Livnat Peer" <lp...@redhat.com>, "Igor Lvovsky" > > > > <ilvov...@redhat.com> > > > > Sent: Monday, December 10, 2012 1:03:48 PM > > > > Subject: Re: Bonding, ifcfg and luck > > > > > > > > On Mon, Dec 10, 2012 at 06:47:58AM -0500, Antoni Segura > > > > Puimedon > > > > wrote: > > > > > Hi all, > > > > > > > > > > I discussed this briefly with Livnat over the phone and > > > > > mentioned > > > > > it to Dan. > > > > > The issue that we have is that, if I understand correctly our > > > > > current > > > > > configNetwork, it could very well be that it works by means > > > > > of > > > > > good > > > > > design with > > > > > a side-dish of luck. > > > > > > > > > > I'll explain myself: > > > > > By design, as documented in > > > > > http://www.kernel.org/doc/Documentation/networking/bonding.txt: > > > > > "All slaves of bond0 have the same MAC address (HWaddr) as > > > > > bond0 > > > > > for all modes > > > > > except TLB and ALB that require a unique MAC address for each > > > > > slave." > > > > > > > > > > Thus, all operations on the slave interfaces after they are > > > > > added > > > > > to the bond > > > > > (except on TLB and ALB modes) that rely on ifcfg will fail > > > > > with a > > > > > message like: > > > > > "Device eth3 has different MAC address than expected, > > > > > ignoring.", > > > > > and no > > > > > ifup/ifdown will be performed. > > > > > > > > > > Currently, we were not noticing this, because we were > > > > > ignoring > > > > > completely > > > > > errors in ifdown and ifup, but > > > > > http://gerrit.ovirt.org/#/c/8415/ > > > > > shed light on > > > > > the matter. As you can see in the following example (bonding > > > > > mode > > > > > 4) the > > > > > behavior is just as documented: > > > > > > > > > > [root@rhel64 ~]# cat /sys/class/net/eth*/address > > > > > 52:54:00:a2:b4:50 > > > > > 52:54:00:3f:9b:28 > > > > > 52:54:00:51:50:49 > > > > > 52:54:00:ac:32:1b <----------------- > > > > > [root@rhel64 ~]# echo "+eth2" > > > > > > /sys/class/net/bond0/bonding/slaves > > > > > [root@rhel64 ~]# echo "+eth3" > > > > > > /sys/class/net/bond0/bonding/slaves > > > > > [root@rhel64 ~]# cat /sys/class/net/eth*/address > > > > > 52:54:00:a2:b4:50 > > > > > 52:54:00:3f:9b:28 > > > > > 52:54:00:51:50:49 > > > > > 52:54:00:51:50:49 <----------------- > > > > > [root@rhel64 ~]# echo "-eth3" > > > > > > /sys/class/net/bond0/bonding/slaves > > > > > [root@rhel64 ~]# cat /sys/class/net/eth*/address > > > > > 52:54:00:a2:b4:50 > > > > > 52:54:00:3f:9b:28 > > > > > 52:54:00:51:50:49 > > > > > 52:54:00:ac:32:1b <----------------- > > > > > > > > > > Obviously, this means that, for example, when we add a bridge > > > > > on > > > > > top of a bond, > > > > > the ifdown, ifup of the bond slaves will be completely > > > > > fruitless > > > > > (although > > > > > luckily that doesn't prevent them from working). > > > > > > > > > > > > Sorry, thi is not obvious to me. > > > > When we change something in a nic, we first take it down (which > > > > break > > > > it > > > > away from the bond), change it, and then take it up again (and > > > > back > > > > to > > > > the bond). > > > > > > > > I did not understand which flow of configuration leads us to > > > > the > > > > "unexpected mac" error. I hope that we can circumvent it. > > > > > > > > > > > > > > > > > > To solve this issue on the ifcfg based operation we could > > > > > either: > > > > > - Continue ignoring these issues and either not do ifup > > > > > ifdown > > > > > for > > > > > bonding > > > > > slaves or catch the specific error and ignore it. > > > > > > > > That's reasonable, for a hack. > > > > > > > > > - Modify the ifcfg files of the slaves after they are > > > > > enslaved to > > > > > reflect the > > > > > MAC addr of /sys/class/net/bond0/address. Modify the ifcfg > > > > > files > > > > > after the > > > > > bond is destroyed to reflect their own addresses as in > > > > > /sys/class/net/ethx/address > > > > > > > > I do not undestand this solution at all... Fixing initscripts > > > > to > > > > expect > > > > the permanent mac address instead of the bond's one makes more > > > > sense > > > > to > > > > me. ( /proc/net/bonding/bond0 has "Permanent HW addr: " ) > > > > > > > > > > > > > > Livnat made me note that this behavior can be a problem to > > > > > the > > > > > anti > > > > > mac-spoofing rules that we add to iptables, as they rely on > > > > > the > > > > > identity device > > > > > -macaddr to work and, obviously, in most bonding modes that > > > > > is > > > > > broken unless > > > > > the device's macaddr is the one chosen for the bond. > > > > > > > > Right. I suppose we can open a bug about it: in-guest bond does > > > > not > > > > work > > > > with mac-no-spoofing. I have a vague memory of discussing this > > > > with > > > > lpeer few months back, but it somehow slipped my mind. > > > > > > > > > > > > > Well, I think that is all for this issue. We should discuss > > > > > which > > > > > is the best > > > > > approach for this before we move on with patches that account > > > > > for > > > > > ifup ifdown > > > > > return information. > > > > > > > > > > Best, > > > > > > > > > > Toni > > > > > > > > > > > > _______________________________________________ > > > vdsm-devel mailing list > > > vdsm-devel@lists.fedorahosted.org > > > https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel > > > > > _______________________________________________ > > vdsm-devel mailing list > > vdsm-devel@lists.fedorahosted.org > > https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel > _______________________________________________ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
