Hi Dave, all,
Thank you for listing the concerns clearly. I understand the concern about
downstream compatibility, and I agree that the blast radius should be
measurable and that out-of-tree plugin maintainers need a practical migration
path. However, I believe these concerns can be addressed concretely without
delaying the discussion until October.
On ABI compatibility, there is no documented policy guaranteeing a stable
libvnet ABI. Across the five release transitions from VPP 24.10 to 26.06, among
functions both exported by libvnet and declared in installed headers, 262
functions were added, 92 were removed, and 64 material prototype changes
affected 62 distinct functions. These included 22 argument-count changes, 35
argument-type changes, and 7 return-type changes. I have attached the detailed
results and methodology.
This does not make downstream compatibility irrelevant. It does mean, however,
that requiring ABI stability specifically for this change would be inconsistent
unless the same requirement were documented and applied to every libvnet change
that removes an exported symbol or modifies an exported prototype.
More importantly, this change improves our control over the libvnet interface.
Today, any non-static symbol may become part of the dynamic interface
accidentally. Hidden visibility with explicit exports turns that accidental
interface into an intentional, reviewable, and CI-checked export set. It
therefore provides a much stronger foundation for defining and maintaining a
stable ABI in the future, should the project decide to make that commitment.
This is also not merely a visibility cleanup. The optimization that motivated
this work produced measurable performance improvements, particularly on ARM.
Measurements showed clear gains across several workloads, with the largest
improvements in latency-sensitive profiles. The purpose of the proposed series
is to obtain those benefits through explicit visibility control while
preserving the symbols that consumers actually require.
Regarding the blast radius, the current patch already checks all in-tree
consumers, and the runtime validation successfully loaded all 104 configured
plugins. The checks have also passed with GCC and Clang on x86, and with Clang
on ARM64.
Florin's concern about global data is already addressed by the current patch:
the export inventory and CI check cover OBJECT and TLS symbols as well as
functions, and the data symbols required by in-tree consumers are explicitly
exported.
The remaining limitation is completeness for out-of-tree consumers, since their
requirements cannot be inferred from the VPP source tree alone. To address
that, I propose exposing the existing ELF-based checker as a standalone command
accepting one or more external plugin binaries. It would report any required
libvnet function, object, or TLS symbol missing from the explicit export set,
without requiring the plugin sources or integration into the VPP build.
Regarding the default, making hidden visibility permanently opt-in would
provide little practical benefit, and the non-default configuration would
probably receive insufficient CI coverage. To simplify review, I can clearly
separate the visibility mechanism, the export checks, the out-of-tree checker,
and the activation of hidden visibility. If a transitional opt-in period is
considered necessary, it should have an explicit exit criterion rather than
leaving the feature permanently untested and disabled by default.
This topic has been public since July 23. Benoit has consistently supported the
hidden-visibility direction and has now suggested discussing it during next
Tuesday's call. Florin raised the global-data and out-of-tree concerns
discussed above.
I have also discussed this offline with Damjan. My understanding from those
exchanges was that he viewed the direction positively, but I do not want to
speak on his behalf and will leave him to state his position if he wishes.
The concrete RFC series was posted on August 26:
https://gerrit.fd.io/r/c/vpp/+/46571
https://gerrit.fd.io/r/c/vpp/+/46572
https://gerrit.fd.io/r/c/vpp/+/46573
I am happy to discuss this next Tuesday, as Benoit suggested. Before the call,
I propose to:
1. Add the standalone out-of-tree plugin checker and document its use.
2. Make the ABI-change analysis and methodology available to the community.
3. Ask reviewers to identify any remaining concrete technical issues on Gerrit
or the mailing list.
The goal of Tuesday's discussion should then be to resolve those remaining
points and reach a decision on the series, rather than defer the subject to
October.
Dave, since you mentioned that you would not be available for the two scheduled
September community meetings, could you please confirm whether you can attend
next Tuesday's discussion? If not, we can address your concerns in writing
beforehand and send you a concise summary of the outcome.
I believe the proposed changes address the practical compatibility concerns,
improve control over what libvnet intentionally exposes, and provide measurable
performance benefits, particularly on ARM. They also avoid introducing an
undocumented ABI requirement applied only to this patch.
Best regards,
Jerome
libvnet exported function prototype analysis
============================================
Scope and method
----------------
Stable VPP releases compared:
24.10, 25.02, 25.06, 25.10, 26.02, 26.06
The analysis was restricted to functions that are both:
- present in the libvnet.so dynamic symbol table; and
- declared in the installed <vnet/...> headers from the official packages.
Function declarations were parsed with Clang. Canonical return and argument
types were compared after resolving typedefs. Anonymous-type source paths were
normalized. Parser artifacts and one ABI-neutral pointer-tag identity change
were excluded from the material prototype-change count.
Aggregated results
------------------
Across the five consecutive release transitions:
- 262 exported and declared functions were added;
- 92 exported and declared functions were removed;
- 64 material prototype-change events affected 62 distinct functions.
The 64 prototype changes comprise:
- 22 argument-count changes;
- 35 argument-type changes;
- 7 return-type changes.
Results by transition
---------------------
Transition Changed prototypes Added Removed
24.10 -> 25.02 13 29 14
25.02 -> 25.06 30 19 11
25.06 -> 25.10 3 37 1
25.10 -> 26.02 5 104 26
26.02 -> 26.06 13 73 40
----------------------------------------------------------------
Total 64 262 92
Between the two endpoints, 24.10 and 26.06, 260 functions were added,
90 were removed, and 58 functions present in both releases had materially
different prototypes. Another 24 declarations were reformulated through
function-type aliases without changing their canonical calling signature.
Representative examples
-----------------------
24.10 -> 25.02:
- Seven bfd_udp_* functions gained a leading _Bool/is_ipv6 argument.
- pg_interface_add_or_get changed from six arguments to an argument struct.
- session_lookup_table_cleanup gained a third argument.
- vnet_dev_port_if_create gained a void * argument.
25.02 -> 25.06:
- Twenty-eight changes relate to worker indices moving from u32 to
clib_thread_index_t (which is typedef'd as u16), including session,
transport, crypto, interface queue, IPFIX, and app-worker functions.
- transport_release_local_endpoint and transport_share_local_endpoint each
gained an argument.
25.06 -> 25.10:
- vnet_dev_rx_queue_alloc changed from three to five arguments.
- vnet_dev_tx_queue_alloc changed from three to four arguments.
- session_dgram_connect_notify changed an argument from u32 to a session
handle union.
25.10 -> 26.02:
- segment_manager_main_init gained an argument.
- virtio_show lost an argument.
- Three endpoint functions changed an argument from u8 to a pointer.
26.02 -> 26.06:
- app_worker_start_listen changed its argument count and shape.
- segment_manager_attach_fifo and segment_manager_detach_fifo lost an
argument.
- vnet_crypto_process_ops gained a chunks argument.
- Several app-worker functions changed return type between int and void.
- vnet_crypto_register_engine changed its return type from u32 to
vnet_crypto_engine_id_t (u8).
Interpretation and limitations
------------------------------
This is a conservative lower bound, not a complete ABI audit: it does not
include exported functions without declarations in installed vnet headers,
object layout changes, global-variable size changes, or plugin libraries.
The results show that the currently exported libvnet surface has changed
regularly across releases. They do not imply that every symbol should be
hidden. They support defining the intended external interface explicitly and,
if the project chooses to guarantee ABI stability in the future, tracking that
interface through documented policy and automated tooling.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#27162): https://lists.fd.io/g/vpp-dev/message/27162
Mute This Topic: https://lists.fd.io/mt/121042664/21656
Group Owner: [email protected]
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/14379924/21656/631435203/xyzzy
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
