On 05/28/2015 08:40 AM, Scott Feldman wrote:
On Thu, May 28, 2015 at 2:42 AM, Jiri Pirko <j...@resnulli.us> wrote:
Mon, May 18, 2015 at 10:19:16PM CEST, da...@davemloft.net wrote:
From: Roopa Prabhu <ro...@cumulusnetworks.com>
Date: Sun, 17 May 2015 16:42:05 -0700
On most systems where you can offload routes to hardware,
doing routing in software is not an option (the cpu limitations
make routing impossible in software).
You absolutely do not get to determine this policy, none of us
do.
What matters is that by default the damn switch device being there
is %100 transparent to the user.
And the way to achieve that default is to do software routes as
a fallback.
I am not going to entertain changes of this nature which fail
route loading by default just because we've exceeded a device's
HW capacity to offload.
I thought I was _really_ clear about this at netdev 0.1
I certainly agree that by default, transparency 1:1 sw:hw mapping is
what we need for fib. The current code is a good start!
I see couple of issues regarding switchdev_fib_ipv4_abort:
1) If user adds and entry, switchdev_fib_ipv4_add fails, abort is
executed -> and, error returned. I would expect that route entry should
be added in this case. The next attempt of adding the same entry will
be successful.
The current behaviour breaks the transparency you are reffering to.
2) When switchdev_fib_ipv4_abort happens to be executed, the offload is
disabled for good (until reboot). That is certainly not nice, alhough
I understand that is the easiest solution for now.
I believe that we all agree that the 1:1 transparency, although it is a
default, may not be optimal for real-life usage. HW resources are
limited and user does not know them. The danger of hitting _abort and
screwing-up the whole system is huge, unacceptable.
So here, there are couple of more or less simple things that I suggest to
do in order to move a little bit forward:
1) Introduce system-wide option to switch _abort to just plain fail.
When HW does not have capacity, do not flush and fallback to sw, but
rather just fail to add the entry. This would not break anything.
Userspace has to be prepared that entry add could fail.
2) Introduce a way to propagate resources to userspace. Driver knows about
resources used/available/potentially_available. Switchdev infra could
be extended in order to propagate the info to the user.
3) Introduce couple of flags for entry add that would alter the default
behaviour. Something like:
NLM_F_SKIP_KERNEL
NLM_F_SKIP_OFFLOAD
Again, this does not break the current users. On the other hand, this
gives new users a leverage to instruct kernel where the entry should
be added to (or not added to).
Any thoughts? Objections?
I don't like these. Breaks transparency and forces the user in a
position of having to know hardware failures modes (unique to each
hardware device). I presented an option d) which avoids this issues;
was it not understood?
Hi Scott,
I understood your proposal. One caveat I had is in response to this,
"Actually, now that I think of it, the device/driver could decide which
related-prefix to evict from HW, if driver/device wanted to have a
sense of which routes are more important to offload than other"
hardware/driver/device shouldn't have a sense of which routes are more
important than others. I think this is where the NLM_F_* flags come in.
If userspace _wants_ to push policy into the kernel about what is
important it can. If it doesn't we get a sensible heuristic that does
a reasonable job offloading rules transparently. This is how we did
L2 and I think that seems to work fairly well. At least for me but,
always interested to hear other use cases though.
Also I guess I'm not seeing the multitude of hardware failure modes. I
see two either the hardware doesn't support the operation or it is out
of resources. Both can be learned if the hardware exports a model of its
capabilities and resources.
Thanks,
John
--
John Fastabend Intel Corporation
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
