On 17/12/2015 19:16, Gleb Smirnoff wrote:
Steven,
On Thu, Dec 17, 2015 at 02:29:02PM +0000, Steven Hartland wrote:
S> I would definitely like to understand more about your concerns and learn
S> from
S> your knowledge in this area, so thanks for that offer, and while it does
S> sound
S> unforgiving I totally understand where you're coming from.
S>
S> Hopefully together we can bring this to a satisfactory conclusion as I
S> would hate
S> for both carp and lagg to stay as broken, 2 years is long enough :D
Ok, let's get technical. CARP and LAGG were not broken for 2 years. They
were working very well in the way they were designed to work. The setup
in the bug 156226 was broken initially.
You may have not read all the detail in the review so you might not have
noticed that I
identified that carp IPv6 NA was broken by r251584 which was committed 2 1/2
years ago. I'm guessing not may people use it for IPv6.
The "link aggregation" itself refers to an aggregation of links between
two logical devices. If you build lagg(4) interface on top of two ports
that are plugged into different switches, you are calling for trouble.
While multiple switches complicates the matter its not the only issue as
you can
reproduce this with a single switch and two nics in LAGG failover mode
with a simple
ifconfig <nic1> down. At this point any traffic entering the switch for
LAGG member
will back-whole instead of being received by the other nic.
It is much more common in networking now to have multiple physical switches
configured as part of bigger logical devices using protocols such as
MLAG, which is
what we're using with Cisco's and Arista's, so not some cheepo network ;-)
All comments in the 156226 from Eugene Grosbein are valid. I would not
repeat them, but ask you to reread them in bugzilla. There was a good
reason why for 2 years committers stayed away from this "bug" and related
patch.
Yes but not confuse the different types, we're talking specifically
about failover mode
here which has no special configuration hence its reliant on the OS
implementation
only.
Nevertheless, someone wants to give a kick to this initially broken
network design and run it somehow. And this "somehow" implies Layer2
upcalling into upper layers to do something, since there is no
established standard layer2 heartbeat packet. I have chatted with
networking gurus at my job, and they said, that they don't know
any decent network equipment that supports such setup. However, they
noticed that Windows is capable for such failover. I haven't yet
learned on how Windows solves the problem. Actually, those who
pushed committing 156226 should have done these investigations.
Probably Windows does exactly the same, sends gratutious ARP or
its IPv6 analog. Or may be does something better like sending
useless L2 datagram, but with a proper source hardware address.
Actually our testing here showed both Windows and Linux worked as
expected and
from my reading doing the GARP / UNA is actually expected in this
situation, for this
very reason.
Okay, what if we want same in FreeBSD as in Windows? Should we do the
following list of evil things:
- put DELAY in context of callout(or in context of any network processing)
- introduce new notions of a link state, or new KPI for link handling
Note that link handling KPI was stable for iver 10 years and satisfied
all the different types of interfaces we support
- create new interface methods
- call into address families supplying an ifnet that doesn't have this AF
instantiated, and then to fix immediate panic putting there a kludge
of "if (foo == NULL) return;"
- etc...
Sorry, I'm putting "etc" here, because tires on details. You would agree
that the whole process of fixing the "bug" was overcoming the problems
that the network stack is not designed for the things that you are
willing to do. Won't you agree?
I am indeed trying to produce feature parity, to prevent the powers that
be throwing
FreeBSD out as the only OS which fails to work as expected in failover
mode, even in the
simple case as described above.
Yes we could apply user land work around but then everyone has to be
aware its need
and to set it up which doesn't sound like the best solution.
Or should we just write a tiny program, that would observe state of
networking ports, and if a port changes state then send a tiny packet
as a bpf(4) write?
This could be done but still means our lagg failover doesn't do what
people would expect.
I'd like to step back for a second and get you feedback on the changes
that where
reverted, which didn't have the DELAY in the callout. What where the
issues as you
saw them? So we don't spam people any more I've reopened the review so
we can
take this there: https://reviews.freebsd.org/D4111
Apologies if these are very obvious to others but clearly those involved
with this
didn't spot them so it would be really nice to learn from this.
Regards
Steve
_______________________________________________
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
