v188.8.131.52 has been released and api-next branch is rebased on top of it. == OpenDataPlane (184.108.40.206) === Summary of Changes ODP v220.127.116.11 is a refresh of ODP, incorporating significant configurability and performance improvements as well as new APIs and API restructures.
==== APIs ===== Symbol `ODP_SHM_NULL` Removed. An invalid `odp_shm_t` has the value `ODP_SHM_INVALID`, consistent with other ODP types. The legacy synonym `ODP_SHM_NULL` is now removed for consistency. ===== New 3GPP Crypto Algorithm Support New support for 3GPP crypto algorithms is added by defining symbols for * `ODP_CIPHER_ALG_KASUMI_F8` * `ODP_CIPHER_ALG_SNOW3G_UEA2` * `ODP_CIPHER_ALG_ZUC_EEA3` In addition new authentication algorithm symbols are defined for * `ODP_AUTH_ALG_KASUMI_F9` * `ODP_AUTH_ALG_SNOW3G_UIA2` * `ODP_AUTH_ALG_ZUC_EIA3` These values are returned as ODP capabilities as well as being accepted in crypto session creation for implementations that indicate support for them. ===== Crypto Capability for Bitwise Operation The new `bit_mode` capability Boolean is added to the `odp_crypto_cipher_capability_t` struct to indicate that an implementation supports operating in bit mode. When operating in bit mode, field offsets and lengths are expressed in terms of bits rather than bytes. However, such lengths must always be specified in multiples of 8. ===== Improved Crypto Spec Documentation The ODP crypto API specification is tightened to specify default values for cipher and authentication algorithms. Also documented when key and IV parameters need to be set. ===== IPsec Extensions IPsec requires "salt" (extra keying material) when the GMAC authentication algorithm is used. To accommodate this the `auth_key_extra` field is added to the `odp_ipsec_crypto_param_t` struct and documentation is added clarifying when this field is needed and how it should be used. ===== Classifier Type Rename The `odp_pmr_t` type name for an invalid value is renamed from `ODP_PMR_INVAL` to `ODP_PMR_INVALID` for consistency with the rest of ODP type names. The old symbol is still available when ODP is configured with `--enable-deprecated`. ===== New API for Packet Event Subtypes The `odp_packet_subtype()` API is added that returns the subtype of a packet event directly. ===== Streamlined Packet Parsing Results The `odp_packet_parse_result()` API is added that returns the result of packet parsing as a single `odp_packet_parse_result_t` struct. This can offer efficiency improvements for applications that need all parse results rather than making individual parse result calls. ===== PktIO Extensions to Support per-Queue Configuration PktIO interfaces support multiple input queues to enable increased parallelism in I/O processing. Previously, all of these input queues were required to belong to the same scheduler group. The `odp_pktin_queue_param_t` struct is now extended with an optional `odp_pktin_queue_param_ovr_t` struct that permits individual pktin queues to be assigned to separate scheduler groups. This may permit improved performance for advanced application use cases. ===== Timer Pool Capabilities The `odp_timer_capability_t` struct is extended to return three additional pieces of information: `max_pools_combined`:: The total number of timer pools that can be created combining different clock sources `max_pools`:: The maximum number of timer pools for a given clock source. `max_timers`:: The maximum number of timers in a single pool. A zero value means number is limited only by available memory. ===== Add Scheduler mix/max/default Priority Functions Three new APIs: `odp_schedule_max_prio()`, `odp_schedule_min_prio()`, and `odp_schedule_default_prio()` are added that return the min, max, and default values specified for the `prio` field in the `odp_schedule_param_t` struct. With the introduction of these scheduling priority functions the previously defined macros (`ODP_SCHED_PRIO_HIGHEST`, `ODP_SCHED_PRIO_NORMAL`, and `ODP_SCHED_PRIO_LOWEST`) are now deprecated and should no longer be used. ===== Specification of `odp_schedule_prio_t` as an `int` Previously, the `odp_schedule_prio_t` type definition was left to each implementation. With the addition of explicit schedule priority ranges, this type is now specified to be an `int` to permit efficient implementation (including inlining) of these functions. ====== New Scheduler APIs The new scheduler APIs `odp_schedule_multi_wait()` and `odp_schedule_multi_no_wait()` are added to provide more efficiently implementable versions of these functions. The existing scheduler APIs remain unchanged. These new APIs can simply provide a fastpath for some applications/implementations as an alternative to specifying a parameter on `odp_schedule_multi()`. ===== Memory Model in `odp_init_global()` The `odp_init_t` parameter passed to `odp_init_global()` is extended to add the `mem_model` field. This field is defined by the new `odp_mem_model_t` struct and is used to specify whether the application will be using a thread (`ODP_MEM_MODEL_THREAD`) or process (`ODP_MEM_MODEL_PROCESS`) memory model. The default is a thread model is used for compatibility with previous levels of ODP. ==== ABI Changes A number of changes to the ODP ABI have also been made in this release to improve application binary portability. ===== Strong Typing for Timer Pools The `odp_timer_pool_t` is now strongly typed. ===== Consistent Initialization The values of the various `ODP_xxx_INVALID` symbols for ODP abstract types in the `odp-linux` reference implementation are now consistently zeros. This reduces errors and improves portability. === Implementation Improvements ==== Configuration File A new configuration file mechanism is introduced that makes use of https://www.hyperrealm.com/libconfig/libconfig_manual.html[libconfig] to enable various runtime ODP parameters to be specified dynamically. Default configuration values for the `odp-linux` reference implementation are contained in the `config/odp-linux-generic.conf` file. Users may override these default values by supplying their own configuration file. At `odp_init_global()` time, if the `ODP_CONFIG_FILE` environment variable is set, this is used to locate the path to the override configuration file. ==== Process Mode Support The `odp-linux` reference implementation now supports applications that run in process mode (`mem_model` = `ODP_MEM_MODEL_PROCESS`) as well as the default thread mode. This support only applies within a single ODP instance, so any `fork()` calls must be done only _after_ `odp_init_global()` has been called to initialize ODP on a root process. ==== Removal of `iQuery` Scheduler The `iQuery` scheduler is removed from the `odp-linux` reference implementation, as it offers no performance advantages and has not seen application use. ==== Number of CPUs The `odp-linux` reference implementation now supports up to 256 CPUs by default (increased from 128). ==== Support for Large Burst Sizes The `odp-linux` reference implementation now supports large burst sizes for both I/O and non-I/O scheduled events. Large bursts (when available) are received directly to the application without any stashing for improved throughput. Burst sizes are configurable via the new configuration file mechanism, as described above. ==== `--without-openssl` Warnings When building `odp-linux` using `--without-openssl` a warning will be issued cautioning that strong cryptography will not be available. ==== Inline Queue Enq/Deq APIs The various enq/deq APIs are now subject to inlining when `odp-linux` is built with `--disable-abi-compat`. ==== Configurable Timer Controls Inline timers are now controlled via a config file option. Timer polling frequency is similarly controlled via the config file. ==== Huge Page Configuration The config file is now used to specify the huge page usage limit. ==== Single and Multi-Consumer/Producer Rings The queue implementation in `odp-linux` now automatically makes use of optimized single and multi-consumer/producer rings to significantly speed up enq/deq processing. ==== `odp_shm_print_all()` Improvements The output from `odp_shm_print_all()` is reworked to provide more useful and comprehensive shared memory usage information in `odp-linux`. ==== IPsec Improvements SA lifetime checking is now more scalable to multiple threads. This significantly reduces overhead for multithreaded IPsec applications. ==== Native Builds When running in non-ABI compatibility mode, `odp-linux` now enables native machine-specific optimizations for the CPU architecture of the local machine. === Validation Test Improvements ==== SCTP Test Packets SCTP test packets are now used in parser testing. SCTP headers are added to ODP and ODP helpers and SCTP checksums are now inserted and verified as part of validation testing. ==== `odp_packet_reset()` Test The packet validation test suite now properly tests `odp_packet_reset()`. === Helper Changes In support of process mode, ODP helper functions have been changed to better match these new capabilities ==== New `enum` The `odph_linux_thread_type_t enum` has been replaced with the new `odp_mem_model_t` type. ==== Helper Options The new `odph_options()` getter function is added that returns applicable options in effect via the new `odph_helper_options_t` struct. This currently includes the memory model (thread or process) that is in use. ==== SCTP Helpers The new helper APIs `odph_sctp_chksum_set()` and `odph_sctp_chksum_verify()` are added to facilitate working with SCTP packet checksums. === Performance Test Improvements ==== Pool Performance A new `odp_pool_perf` test has been added that stress-tests ODP pool functions in a multithreaded environment to generate performance statistics. ==== Scheduler Performance A new `odp_sched_perf` test has been added that stress-tests the scheduler in a multithreaded environment. ==== CPU Performance A new `odp_cpu_bench` performance test has been added that runs compute-intensive packet operations in a multithreaded environment and prints the observed maximum throughput for each thread. === Example Improvements ==== Classifier Example changes The `odp_classifier` example program now uses a reduced number of threads by default to reduce elapsed run time. `ODP_THREAD_COUNT_MAX` is also now used as the max worker count. ==== Generator Improvements The `odp_generator` example has numerous cleanups and performance improvements. ==== IPsec Example The `odp_ipsec` example now properly stops and closes pktio devices on exit. ==== Packet Dumping A new `odp_packet_dump` example is added that prints received packets to the terminal. This is useful for debugging packet I/O interfaces. ==== Sysinfo Example A new `odp_sysinfo` example is provided that prints system information. Useful for checking the ODP environment during debugging. This includes providing detailed information about the various crypto facilities supported, as well as the feature flags used at build time (_e.g.,_ if the binary was built with ARMv8.0 or ARMv8.1 instructions). ==== Traffic Manager Example The traffic manager example now properly destroys all TM queues it creates for improved reliability. It also now always prints a proper termination summary message. === Bug Fixes ==== Numbered Bugs/Issues ===== https://bugs.linaro.org/show_bug.cgi?id=3983[Bug 3983] Compile fails on OpenSuSE 42.2 Leap with error: negative width in bit field '__error_if_negative' ===== https://bugs.linaro.org/show_bug.cgi?id=3989[Bug 3989] odp_system_info_init() issues ===== https://bugs.linaro.org/show_bug.cgi?id=3999[Bug 3999] IPsec antireplay check drops packets when sequence number jumps. ===== https://bugs.linaro.org/show_bug.cgi?id=4002[Bug 4002] IPsec SA creation must fail for ESN-enabled SAs ===== https://bugs.linaro.org/show_bug.cgi?id=4013[Bug 4013] Per-SA IPv4 ID allocation may cause duplicate IDs. ===== https://bugs.linaro.org/show_bug.cgi?id=4017[Bug 4017] Unexpected IP ID causes IPsec API validation to fail ===== https://github.com/Linaro/odp/issues/662[Issue 662] rte_mempool_ops_alloc() is not dpdk api ==== Unnumbered Bugs/Issues * Fixed enq/deq issues encountered on architectures with weak memory ordering. * Return 0 from `odp_sys_huge_page_size_all()` if hugepages are not supported/detected. Tests modified to not treat this as an error. * Set `ODP_CACHE_LINE_SIZE` to 128 on ppc64le systems. * iplookuptable fix putting values into table * DPDK pktio support now works properly across multiple ODP instances. * Zero timer pool memory on reserve (fixes timer failures due to uninitialized variables). * `-march=native` disabled for `clang`. This fixes a known issue with recent levels of clang. === Known Issues ==== https://bugs.linaro.org/show_bug.cgi?id=3998[Bug 3998] IPsec extended sequence number support is missing ==== https://bugs.linaro.org/show_bug.cgi?id=4014[Bug 4014] Separate IP ID allocation for transport and tunnel mode SAs may cause duplicate IDs ==== https://bugs.linaro.org/show_bug.cgi?id=4018[Bug 4018] Unexpected IV causes IPsec API validation to fail ==== https://bugs.linaro.org/show_bug.cgi?id=4040[Bug 4040] Clang build fails on Ubuntu 18.04 Best regards, Maxim.