Main goal of this patchset is to address https://bugs.dpdk.org/show_bug.cgi?id=1957 but it also handles other recently stabilized symbols and has some minor fixes:
- Patch 1 - Fix RTE_VERSION_EXPERIMENTAL_SYMBOL macro on clang. - Patch 2 - Allow function versioning inside drivers. - Patch 3 - Version the function symbols stabilized in https://git.dpdk.org/dpdk/commit/?id=e8cab133645f5466ef75e511629add43b68a5027 - Patch 4 - Introduce versioning macros for global variable symbols. - Patch 5 - Version the function and variable symbols stabilized in https://git.dpdk.org/dpdk/commit/?id=4ee2f5c1cedf9ee7f39afa667f71b07f4004ba5c Issue is still not fully fixed for stabilized global variables: rte_flow_dynf_metadata_offs and rte_flow_dynf_metadata_mask. Patch 4 and 5 address the bug for these global variables, by providing a single storage for both EXPERIMENTAL and DPDK_26 variable symbol versions. This is achieved through symbol aliasing. But this solution is limited only to executables compiled with clang. clang and gcc have a different default behavior regarding relocations of global variables exposed by shared libraries. With clang, R_X86_64_GLOB_DAT relocations are generated for executables: $ readelf -sW build-26.07/lib/librte_ethdev.so | grep rte_flow_dynf_metadata_offs 113: 00000000000ea4c0 4 OBJECT GLOBAL DEFAULT 24 rte_flow_dynf_metadata_offs@@DPDK_26 116: 00000000000ea4c0 4 OBJECT GLOBAL DEFAULT 24 rte_flow_dynf_metadata_offs@EXPERIMENTAL 970: 00000000000ea4c0 4 OBJECT LOCAL DEFAULT 24 rte_flow_dynf_metadata_offs_impl 1212: 00000000000ea4c0 4 OBJECT LOCAL DEFAULT 24 rte_flow_dynf_metadata_offs_v26 1325: 00000000000ea4c0 4 OBJECT LOCAL DEFAULT 24 rte_flow_dynf_metadata_offs_exp 1415: 00000000000ea4c0 4 OBJECT GLOBAL DEFAULT 24 rte_flow_dynf_metadata_offs@@DPDK_26 1705: 00000000000ea4c0 4 OBJECT GLOBAL DEFAULT 24 rte_flow_dynf_metadata_offs@EXPERIMENTAL $ readelf -rW build-26.07/drivers/librte_net_mlx5.so | grep rte_flow_dynf_metadata_offs 0000000003ed5f18 0000001600000006 R_X86_64_GLOB_DAT 0000000000000000 rte_flow_dynf_metadata_offs@DPDK_26 + 0 $ readelf -rW build-25.11/app/dpdk-testpmd | grep rte_flow_dynf_metadata_offs --> 000000000028ef70 0000011300000006 R_X86_64_GLOB_DAT 0000000000000000 rte_flow_dynf_metadata_offs@EXPERIMENTAL + 0 With gcc, R_X86_64_COPY relocations are generated: $ readelf -sW build-26.07/lib/librte_ethdev.so | grep rte_flow_dynf_metadata_offs 113: 00000000000e74e0 4 OBJECT GLOBAL DEFAULT 24 rte_flow_dynf_metadata_offs@@DPDK_26 116: 00000000000e74e0 4 OBJECT GLOBAL DEFAULT 24 rte_flow_dynf_metadata_offs@EXPERIMENTAL 1471: 00000000000e74e0 4 OBJECT LOCAL DEFAULT 24 rte_flow_dynf_metadata_offs_impl 2134: 00000000000e74e0 4 OBJECT LOCAL DEFAULT 24 rte_flow_dynf_metadata_offs_v26 2247: 00000000000e74e0 4 OBJECT LOCAL DEFAULT 24 rte_flow_dynf_metadata_offs_exp 2337: 00000000000e74e0 4 OBJECT GLOBAL DEFAULT 24 rte_flow_dynf_metadata_offs@@DPDK_26 2627: 00000000000e74e0 4 OBJECT GLOBAL DEFAULT 24 rte_flow_dynf_metadata_offs@EXPERIMENTAL $ readelf -rW build-26.07/drivers/librte_net_mlx5.so | grep rte_flow_dynf_metadata_offs 00000000046dbef0 0000001600000006 R_X86_64_GLOB_DAT 0000000000000000 rte_flow_dynf_metadata_offs@DPDK_26 + 0 $ readelf -rW build-25.11/app/dpdk-testpmd | grep rte_flow_dynf_metadata_offs --> 000000000029b540 000001d200000005 R_X86_64_COPY 000000000029b540 rte_flow_dynf_metadata_offs@EXPERIMENTAL + 0 With copy relocations (testpmd linked through gcc) the following happens: - When variable symbol (with EXPERIMENTAL version) gets resolved inside executable, global variable gets copied from read-only data to executable's BSS section. Executable will access this variable through BSS. - When variable symbol (with DPDK_26 version) gets resolved inside a library, global variable is accessed indirectly through GOT. It is stored inside BSS section of the shared library. So executable and libraries refer to different storage, eventually leading to inconsistent runtime behavior. Problems only appears when executable and library require different versions of global variable symbol. If testpmd from 26.07 is used with libraries from 26.07, GOT entry for these variables will point to copied variable. Without copy relocations (testpmd linked through clang) both executable and libraries access the global variable indirectly through GOT. Runtime behavior is consistent, regardless of the mix of variable symbol versions. The only other solution I could find was to use dlsym() inside libraries to dynamically resolve the location rte_flow_dynf_metadata_offs and rte_flow_dynf_metadata_mask, but this solution sounds like an overkill. Essentially this would require moving to getter/setter functions for these variables inside the library. I would appreciate any feedback or suggestions if anybody had encountered a similar issue before. Dariusz Sosnowski (5): eal: fix macro for versioned experimental symbol drivers: support function versioning net/mlx5: fix stabilized function versions eal: support aliases for versioned variable symbols ethdev: fix promoted flow metadata symbols buildtools/gen-version-map.py | 11 ++++++++++ drivers/meson.build | 8 +++++++ drivers/net/mlx5/meson.build | 2 ++ drivers/net/mlx5/mlx5_driver_event.c | 22 ++++++++++++++----- drivers/net/mlx5/mlx5_flow.c | 18 ++++++++++----- lib/eal/common/eal_export.h | 24 +++++++++++++++++++- lib/ethdev/meson.build | 2 ++ lib/ethdev/rte_flow.c | 33 ++++++++++++++++++---------- 8 files changed, 96 insertions(+), 24 deletions(-) -- 2.47.3
