Hi Maxime,
On 3/23/2026 11:27 AM, Maxime Leroy wrote:
Hi Vladimir,
On Sun, Mar 22, 2026 at 4:42 PM Vladimir Medvedkin
<[email protected]> wrote:
This series adds multi-VRF support to both IPv4 and IPv6 FIB paths by
allowing a single FIB instance to host multiple isolated routing domains.
Currently FIB instance represents one routing instance. For workloads that
need multiple VRFs, the only option is to create multiple FIB objects. In a
burst oriented datapath, packets in the same batch can belong to different
VRFs, so
the application either does per-packet lookup in different FIB instances or
regroups packets by VRF before lookup. Both approaches are expensive.
To remove that cost, this series keeps all VRFs inside one FIB instance and
extends lookup input with per-packet VRF IDs.
The design follows the existing fast-path structure for both families. IPv4 and
IPv6 use multi-ary trees with a 2^24 associativity on a first level (tbl24). The
first-level table scales per configured VRF. This increases memory usage, but
keeps performance and lookup complexity on par with non-VRF implementation.
Thanks for the RFC. Some thoughts below.
Memory cost: the flat TBL24 replicates the entire table for every VRF
(num_vrfs * 2^24 * nh_size). With 256 VRFs and 8B nexthops that is
32 GB for TBL24 alone. In grout we support up to 256 VRFs allocated
on demand -- this approach forces the full cost upfront even if most
VRFs are empty.
Yes, increased memory consumption is the
trade-off.WemakethischoiceinDPDKquite often,such as pre-allocatedmbufs,
mempoolsand many other stuff allocated in advance to gain performance.
For FIB, I chose to replicate TBL24 per VRF for this same reason.
And, as Morten mentioned earlier, if memory is the priority, a table
instance per VRF allocated on-demand is still supported.
The high memory cost stems from TBL24's design: for IPv4, it was
justified by the BGP filtering convention (no prefixes more specific
than /24 in BGPv4 full view), ensuring most lookups hit with just one
random memory access. For IPv6, we should likely switch to a 16-bit TRIE
scheme on all layers. For IPv4, alternative algorithms with smaller
footprints (like DXR or DIR16-8-8, as used in VPP) may be worth
exploring if BGP full view is not required for those VRFs.
Per-packet VRF lookup: Rx bursts come from one port, thus one VRF.
Mixed-VRF bulk lookups do not occur in practice. The three AVX512
code paths add complexity for a scenario that does not exist, at
least for a classic router. Am I missing a use-case?
That's not true, you're missing out on a lot of established core use
cases that are at least 2 decades old:
- VLAN subinterface abstraction. Each subinterface may belong to a
separate VRF
- MPLS VPN
- Policy based routing
I am not too familiar with DPDK FIB internals, but would it be
possible to keep a separate TBL24 per VRF and only share the TBL8
pool?
it is how it is implemented right now with one note - TBL24 are pre
allocated.
Something like pre-allocating an array of max_vrfs TBL24
pointers, allocating each TBL24 on demand at VRF add time,
and you suggesting to allocate TBL24 on demand by adding an extra
indirection layer. Thiswill leadtolowerperformance,whichIwouldliketo avoid.
and
having them all point into a shared TBL8 pool. The TBL8 index in
TBL24 entries seems to already be global, so would that work without
encoding changes?
Going further: could the same idea extend to IPv6? The dir24_8 and
trie seem to use the same TBL8 block format (256 entries, same
(nh << 1) | ext_bit encoding, same size). Would unifying the TBL8
allocator allow a single pool shared across IPv4, IPv6, and all
VRFs? That could be a bigger win for /32-heavy and /128-heavy tables
and maybe a good first step before multi-VRF.
So, you are suggesting merging IPv4 and IPv6 into a single unified FIB?
I'm not sure how this can be a bigger win, could you please elaborate
more on this?
Regards,
Maxime Leroy
Vladimir Medvedkin (4):
fib: add multi-VRF support
fib: add VRF functional and unit tests
fib6: add multi-VRF support
fib6: add VRF functional and unit tests
app/test-fib/main.c | 257 ++++++++++++++++++++++--
app/test/test_fib.c | 298 +++++++++++++++++++++++++++
app/test/test_fib6.c | 319 ++++++++++++++++++++++++++++-
lib/fib/dir24_8.c | 241 ++++++++++++++++------
lib/fib/dir24_8.h | 255 ++++++++++++++++--------
lib/fib/dir24_8_avx512.c | 420 +++++++++++++++++++++++++++++++--------
lib/fib/dir24_8_avx512.h | 80 +++++++-
lib/fib/rte_fib.c | 158 ++++++++++++---
lib/fib/rte_fib.h | 94 ++++++++-
lib/fib/rte_fib6.c | 166 +++++++++++++---
lib/fib/rte_fib6.h | 88 +++++++-
lib/fib/trie.c | 158 +++++++++++----
lib/fib/trie.h | 51 +++--
lib/fib/trie_avx512.c | 225 +++++++++++++++++++--
lib/fib/trie_avx512.h | 39 +++-
15 files changed, 2453 insertions(+), 396 deletions(-)
--
2.43.0
--
Regards,
Vladimir
