Re: [PATCH net-next v2] net: ipv6: avoid overhead when no custom FIB rules are installed

[David Miller]

From: Vincent Bernat 
Date: Tue,  8 Aug 2017 20:23:49 +0200

> If the user hasn't installed any custom rules, don't go through the
> whole FIB rules layer. This is pretty similar to f4530fa574df (ipv4:
> Avoid overhead when no custom FIB rules are installed).
> 
> Using a micro-benchmark module [1], timing ip6_route_output() with
> get_cycles(), with 40,000 routes in the main routing table, before this
> patch:
> 
> min=606 max=12911 count=627 average=1959 95th=4903 90th=3747 50th=1602 
> mad=821
> table=254 avgdepth=21.8 maxdepth=39
> value │ ┊count
>   600 │▒▒▒ 199
>   880 │▒▒▒  43
>  1160 │▒▒▒  48
>  1440 │▒▒▒░░░   43
>  1720 │░░░  59
>  2000 │▒▒▒  50
>  2280 │▒▒░░░26
>  2560 │▒▒░  31
>  2840 │▒▒   28
>  3120 │▒░░  17
>  3400 │▒░░░ 17
>  3680 │░ 8
>  3960 │░░   11
>  4240 │░░6
>  4520 │░░░   6
>  4800 │░░░   9
> 
> After:
> 
> min=544 max=11687 count=627 average=1776 95th=4546 90th=3585 50th=1227 
> mad=565
> table=254 avgdepth=21.8 maxdepth=39
> value │ ┊count
>   540 │201
>   800 │▒63
>  1060 │▒░   68
>  1320 │▒▒▒░░39
>  1580 │▒▒░░ 32
>  1840 │▒▒   32
>  2100 │▒▒░░░34
>  2360 │▒▒░░ 33
>  2620 │▒▒   26
>  2880 │▒░░  22
>  3140 │  9
>  3400 │░ 8
>  3660 │░ 9
>  3920 │░░8
>  4180 │░░░   8
>  4440 │░░░   8
> 
> At the frequency of the host during the bench (~ 3.7 GHz), this is
> about a 100 ns difference on the median value.
> 
> A next step would be to collapse local and main tables, as in
> 0ddcf43d5d4a (ipv4: FIB Local/MAIN table collapse).
> 
> [1]: 
> https://github.com/vincentbernat/network-lab/blob/master/lab-routes-ipv6/kbench_mod.c
> 
> Signed-off-by: Vincent Bernat 
> Reviewed-by: Jiri Pirko 

Looks great, applied, thanks!

Re: [PATCH net-next v2] net: ipv6: avoid overhead when no custom FIB rules are installed

[David Ahern]

On 8/8/17 12:23 PM, Vincent Bernat wrote:
> If the user hasn't installed any custom rules, don't go through the
> whole FIB rules layer. This is pretty similar to f4530fa574df (ipv4:
> Avoid overhead when no custom FIB rules are installed).
> 
> Using a micro-benchmark module [1], timing ip6_route_output() with
> get_cycles(), with 40,000 routes in the main routing table, before this
> patch:
...
> At the frequency of the host during the bench (~ 3.7 GHz), this is
> about a 100 ns difference on the median value.
> 
> A next step would be to collapse local and main tables, as in
> 0ddcf43d5d4a (ipv4: FIB Local/MAIN table collapse).
> 
> [1]: 
> https://github.com/vincentbernat/network-lab/blob/master/lab-routes-ipv6/kbench_mod.c
> 
> Signed-off-by: Vincent Bernat 
> Reviewed-by: Jiri Pirko 
> ---
>  include/net/netns/ipv6.h |  1 +
>  net/ipv6/fib6_rules.c| 40 +++-
>  net/ipv6/route.c |  1 +
>  3 files changed, 29 insertions(+), 13 deletions(-)
> 

LGTM.

Acked-by: David Ahern 

[PATCH net-next v2] net: ipv6: avoid overhead when no custom FIB rules are installed

[Vincent Bernat]

If the user hasn't installed any custom rules, don't go through the
whole FIB rules layer. This is pretty similar to f4530fa574df (ipv4:
Avoid overhead when no custom FIB rules are installed).

Using a micro-benchmark module [1], timing ip6_route_output() with
get_cycles(), with 40,000 routes in the main routing table, before this
patch:

min=606 max=12911 count=627 average=1959 95th=4903 90th=3747 50th=1602 
mad=821
table=254 avgdepth=21.8 maxdepth=39
value │ ┊count
  600 │▒▒▒ 199
  880 │▒▒▒  43
 1160 │▒▒▒  48
 1440 │▒▒▒░░░   43
 1720 │░░░  59
 2000 │▒▒▒  50
 2280 │▒▒░░░26
 2560 │▒▒░  31
 2840 │▒▒   28
 3120 │▒░░  17
 3400 │▒░░░ 17
 3680 │░ 8
 3960 │░░   11
 4240 │░░6
 4520 │░░░   6
 4800 │░░░   9

After:

min=544 max=11687 count=627 average=1776 95th=4546 90th=3585 50th=1227 
mad=565
table=254 avgdepth=21.8 maxdepth=39
value │ ┊count
  540 │201
  800 │▒63
 1060 │▒░   68
 1320 │▒▒▒░░39
 1580 │▒▒░░ 32
 1840 │▒▒   32
 2100 │▒▒░░░34
 2360 │▒▒░░ 33
 2620 │▒▒   26
 2880 │▒░░  22
 3140 │  9
 3400 │░ 8
 3660 │░ 9
 3920 │░░8
 4180 │░░░   8
 4440 │░░░   8

At the frequency of the host during the bench (~ 3.7 GHz), this is
about a 100 ns difference on the median value.

A next step would be to collapse local and main tables, as in
0ddcf43d5d4a (ipv4: FIB Local/MAIN table collapse).

[1]: 
https://github.com/vincentbernat/network-lab/blob/master/lab-routes-ipv6/kbench_mod.c

Signed-off-by: Vincent Bernat 
Reviewed-by: Jiri Pirko 
---
 include/net/netns/ipv6.h |  1 +
 net/ipv6/fib6_rules.c| 40 +++-
 net/ipv6/route.c |  1 +
 3 files changed, 29 insertions(+), 13 deletions(-)

diff --git a/include/net/netns/ipv6.h b/include/net/netns/ipv6.h
index abdf3b40303b..0e50bf3ed097 100644
--- a/include/net/netns/ipv6.h
+++ b/include/net/netns/ipv6.h
@@ -65,6 +65,7 @@ struct netns_ipv6 {
unsigned int ip6_rt_gc_expire;
unsigned longip6_rt_last_gc;
 #ifdef CONFIG_IPV6_MULTIPLE_TABLES
+   bool fib6_has_custom_rules;
struct rt6_info *ip6_prohibit_entry;
struct rt6_info *ip6_blk_hole_entry;
struct fib6_table   *fib6_local_tbl;
diff --git a/net/ipv6/fib6_rules.c b/net/ipv6/fib6_rules.c
index 2f29e4e33bd3..b240f24a6e52 100644
--- a/net/ipv6/fib6_rules.c
+++ b/net/ipv6/fib6_rules.c
@@ -63,19 +63,32 @@ unsigned int fib6_rules_seq_read(struct net *net)
 struct dst_entry *fib6_rule_lookup(struct net *net, struct flowi6 *fl6,
   int flags, pol_lookup_t lookup)
 {
-   struct fib_lookup_arg arg = {
-   .lookup_ptr = lookup,
-   .flags = FIB_LOOKUP_NOREF,
-   };
-
-   /* update flow if oif or iif point to device enslaved to l3mdev */
-   l3mdev_update_flow(net, flowi6_to_flowi(fl6));
-
-   fib_rules_lookup(net->ipv6.fib6_rules_ops,
-flowi6_to_flowi(fl6), flags, );
-
-   if (arg.result)
-   return arg.result;
+   if