OpenDataPlane (1.15.0.0)

New Features

ODP v1.15.0.0 continues the preview of Tiger Moth, introducing new APIs and extensions, as well as numerous bug fixes and functional improvements.

Deprecation Framework

To permit smoother evolution of the ODP API specification, a deprecation framework is introduced to permit controlled deprecation.

When an ODP API or defined struct parameter is deprecated, ODP validation tests will be updated to no longer use that API and instead use the replacement API. By default, attempts to compile with the older API/feature will fail and applications wishing to move to the new ODP release should be updated to use the replacement API. To permit evaluation of new ODP releases in advance of such updating, however, ODP supports the configure option --enable-deprecated, which makes the obsolete APIs visible again.

This feature will be used on a case-by-case basis and documented in the release notes for each release that introduces replacement API(s). In general the deprecated forms will not be maintained for more than a single release cycle. After that they will no longer be present in the API specification and the replacement forms must be used to compile with that level of the API specification.

APIs

A number of new and refined APIs are introduced in crypto, packet parsing, and queue configuration:

Crypto Enhancements

The ODP crypto APIs receive several enhancements in this release:

New Authentication Algorithms

Additional enumerations added for HMAC-SHA-1 and HMAC-SHA-512 authentication.

Deprecated Cipher/Authentication Algorithms

The following cipher/authentication algorithms have been deprecated in favor of newer replacement algorithms:

  • ODP_CIPHER_ALG_AES128_CBC is replaced by ODP_CIPHER_ALG_AES_CBC

  • ODP_CIPHER_ALG_AES128_GCM is replaced by ODP_CIPHER_ALG_AES_GCM

  • ODP_AUTH_ALG_MD5_96 is replaced by ODP_AUTH_ALG_MD5_HMAC

  • ODP_AUTH_ALG_SHA256_128 is replaced by ODP_AUTH_ALG_SHA256_HMAC

  • ODP_AUTH_ALG_AES128_GCM1 is replaced by `ODP_AUTH_ALG_AES_GCM

Deprecated Name for Crypto Parameter struct

odp_crypto_op_params_t is deprecated in favor of odp_crypto_op_param_t for consistency with other ODP param structs.

Digest Length Session Parameter

odp_crypto_session_param_t adds the field auth_digest_len to permit specification of the authentication digest length to be used for this session. The odp_crypto_auth_capability() API returns the list of supported digest lengths.

Additional Authentication Data (AAD)

The odp_crypto_op_param_t struct adds an optional pointer and length for AAD information. This allows applications to specify AAD information from the list of supported lengths provided by odp_crypto_auth_capability().

Packet Range Data

The former odp_crypto_data_range_t type is deprecated and renamed to odp_packet_data_range_t since it can be used to specify ranges for other than crypto purposes.

Parser Configuration

Applications may now specify the maximum packet layer of interest. For example, a router may not care about anything beyond ISO Layer 3 in packets. This can be used by ODP implementations to control the depth of packet parsing needed by the application and may allow greater efficiency, especially in software implementations.

This is controlled by a new odp_pktio_parser_layer_t enum that is part of the new odp_pktio_parser_config_t struct that is added to the odp_pktio_config_t struct used by the odp_pktio_config() API. The supported parser layers are also returned in this struct as part of the odp_pktio_capability_t struct returned by the odp_pktio_capability() API.

Queue Size Parameter

The odp_queue_capability_t struct returned by the odp_queue_capability() API is enhanced as follows: * The max_queues field is now defined to return the maximum number of event queues of any type. * New sub-structs (plain and sched) are added that detail the max_num of queues of that type supported as well as the new field max_size that specifies the maximum number of elements that each queue of this type can store. A value of zero for max_num indicates no fixed limit.

In addition, the odp_queue_param_t passed to odp_queue_create() now adds a size field to allow applications to specify the minimum number of events that this queue should be able to store. A value of zero specified requests that the implementation default limits be used.

The ODP examples have been updated to show this configuration, for example, the l2fwd_simple example does Layer 2 forwarding and hence doesn’t need packets to be parsed beyond Layer 2. So prior to opening an interface via odp_pktio_open() the configuration processing now includes code of the form:

odp_pktio_config_init(&config);
config.parser.layer = ODP_PKTIO_PARSER_LAYER_L2;
odp_pktio_config(pktio, &config);

This serves as a hint to the ODP implementation that parsing beyond Layer 2 is not required on this interface.

Removal of odp_time_to_u64() API

The odp_time_to_u64() API is removed without deprecation in this release. This is redundant since the odp_time_to_ns() API already returns a uint64_t value for odp_time_t and can be used for the same purpose.

New odp_sys_info_print() API

For debugging / diagnostic use, the odp_sys_info_print() API is added. This prints implementation defined information about the system and ODP environment and may contain information about CPUs, memory, and other hardware configuration.

Helpers

Linux Helpers

ODP helper functions fall into two categories: system-independent, and those that rely on Linux. For backwards compatibility, these have been reorganized into a new helper/linux directory that is installed only when the --enable-helper-linux configure option is specified.

odp-linux Performance

Scheduler Improvements

The default scheduler is streamlined by using precomputed weight tables and separated priority queues for different scheduling groups. This improves both latency and scalability.

Also included in this change, the default number of scheduling groups is lowered from 256 to 32, which experience has shown is sufficient for most applications.

The scheduler fairness checks are also enhanced to avoid low priority event starvation if CONFIG_BURST_SIZE is set to 1.

odp_linux Crypto Improvements

  • Errors in implicit packet copy operations performed during crypto operation are now handled properly.

  • If odp_crypto_session_create() fails, proper cleanup is now performed to avoid memory leaks.

  • The auth digest len is now used when calculating HMACs, and full-length digests are now added to supported SHA capabilities.

  • Numerous crypto functions have been rewritten to use the OpenSSL EVP_ functions for improved efficiency and accuracy.

odp-linux Packet Improvements

  • The packet parser now recognizes ICMPv6 packets in addition to ICMPv4.

odp-linux PktIO Improvements

  • The ethtool and socket drivers add additional initializations to avoid valgrind warnings.

  • The dpdk driver is upgraded to DPDK 17.02 and is now also built with the --whole-archive option, which means that PMDs no longer need to be linked individually but are automatically included in builds that make use of DPDK pktio support.

odp-linux Time Improvements

  • The ODP Time APIs are now more efficiently handled by replacing the previous Linux timespec with simple nanoseconds (ns) and using native hardware time counters in x86 and ARM environments, when available.

odp-linux Traffic Manager Improvements

  • Weighting is now handled properly when weight=1 is specified. Previously this caused an overflow that distorted packet priorities.

Miscellaneous Fixes and Improvements

Native Clang build on ARMv8

ARMv8 compilation now works properly when using the clang compiler.

Test Improvements

  • The odp_scheduling performance test now handles dequeueing from potentially concurrent queues properly.

  • The traffic manager and time tests are now invoked via scripts to better account for load-sensitivity when running regressions in CI environments.

  • The l2fwd test now supports an additional parameter to specify the number of scheduling groups to use in the test.

  • The odp_sched_latency performance test now handles queue draining in a robust manner.

  • The crypto validation tests now properly check and validate message digests, and include negative tests to verify that incorrect digests are properly flagged. Note that this may expose omissions in other ODP implementations that had previously passed this validation test.

  • The crypto validation test now adds an explicit test for the NULL cipher, as well as HMAC-SHA-1 and HMAC-SHA-512.

  • The pktio_ipc test now properly cleans up only its own temp files.

Examples Improvements

  • The odp_generator example now properly honors stop criteria based on number of packets sent.

  • The odp_generator example now properly flushes output messages

  • The odp_generator example now supports port configuration.

Bug Fixes

Bug 2416

example/generator/odp_generator.c contains todo items

Bug 2779

Error handling issues (CHECKED_RETURN)

Bug 2826

Unchecked return in pool.c

Bug 2831

Unchecked return in mmap_unmap_sock()

Bug 2852

ODP_STATIC_ASSERT() fails when used by C++

Bug 2872

odp_pktio_ordered_run.sh: line 34: integer expression expected

Bug 2881

load sensitive tests fail on CI

Bug 2895

The odp_crypto_operation() routine does not work with multi-segment packets.

Bug 2908

Packet validation test fails if implementation does not limit packet length

Bug 2910

odph_iplookup_table_put_value() uses overlapping pointer addresses

Bug 2933

Miss to call unlock if there are some errors happen in loopback_send() function

Bug 2940

odp_packet_seg_t fails ABI compatibility between odp-linux and odp-dpdk

Bug 2942

Compilation fails using clang 4.0.0

Bug 2952

doxygen errors and travis does not catch them

Bug 2969

TM validation test does find traffic_mngr_main

Bug 2974

odp_rwlock_read_trylock() fails if lock is already held for read access

Bug 3002

Packet order lost when enqueuing to pktout queue

Bug 3003

AES-GCM returns 'valid' tag when checking invalid tag

Bug 3013

Various failures if CONFIG_PACKET_MAX_SEGS is set to 1

Bug 3017

Travis: time main test out of boundaries

Bug 3027

Compilation failures using GCC 7 series

Bug 3039

Socket pktio recv fails on large number of packet

Known Issues

Bug 3024

odp_traffic_mngr example is broken

Bug 3026

pktio_ips_run test can fail due to segfault

OpenDataPlane (1.14.0.0)

New Features

APIs

ODP v1.14.0.0 represents another preview of the Tiger Moth release series and introduces new APIs and extensions.

Packet References

Packet references are a lightweight mechanism to allow applications to create and manipulate different "views" of packets. These views consist of shared bytes common to all references created on the same odp_packet_t, possibly prefixed with unique headers that are private to each reference. Five new APIs are introduced to enable this feature:

odp_packet_ref_static()

Static references allow a single packet to have multiple "aliases", all of which must be treated as read only. This is useful, for example, to retain a reference to a packet being transmitted to support retransmit processing, since the actual storage represented by a packet is not released until all references to it have been freed via odp_packet_free() calls.

odp_packet_ref()

Dynamic references differ from static references is that they permit the individual odp_packet_t handles to have unique prefixes created via odp_packet_push_head() or odp_packet_extend_head() calls. This can be used, for example, to support multicasting packets to different destinations by creating packets that consist of unique headers followed by a common shared payload.

odp_packet_ref_pkt()

Similar to odp_packet_ref(), but creates a dynamic reference by prepending a preexisting header onto another packet.

odp_packet_has_ref()

Returns a boolean indicator of whether a given odp_packet_t has other references that share bytes with this packet.

odp_packet_unshared_len()

Returns the number of unshared bytes accessible through a given odp_packet_t handle. These are the only bytes that should be changed. All other bytes in the packet should be treated as read only as they are shared with other odp_packet_t handles.

Note that while the packet reference APIs and associated validation tests are present in this release, the odp-linux reference implementation currently implements references via packet copying. A zero-copy implementation of packet references is planned to be part of the ODP v1.15.0.0 release.

ABI Specification

ODP has supported an Application Binary Interface (ABI) that permits applications to be generated that are binary portable between select ODP implementations. This is now formalized with the addition of structures that permit ABI specifications to be defined that are shared between ODP implementations. ODP provides a default ABI specification that is the same as was provided before. This change means that additional ABIs may be defined that are optimized to collections of platforms that share an Instruction Set Architecture (ISA) and subscribe to them.

ABI specifications live in the include/odp/arch/…​ directory.

Instance Query (iQuery) Scheduler

Adding to the range of scheduler implementations supported by odp-linux, a new scheduler, the instance query scheduler, is available by specifying --enable-schedule-iquery at configure time.

This scheduler uses sparse bitmaps and offers scalability advantages when dealing with large numbers of schedule queues. It otherwise offers performance comparable to the default ODP scheduler implementation.

Helpers

Helpers have been reorganized to provide better independence from odp-linux and to make them more useful with other ODP implementations. These reorganizations are functionally transparent to ODP users but should simplifying packaging and porting to other ODP implementations. This includes adding the "umbrella" include file odph_api.h which can be used to include all helper API definitions in an application.

Classifier Improvements

The odp-linux reference implementation now supports Pattern Matching Rules (PMRs) for IPv6 addresses, inner and outer VLAN IDs, and inner and outer Ethernet types.

Performance

Improved inlining for embedded applications

The odp-linux reference implementation now offers improved inlining support for ODP APIs for applications compiled against ODP configured with the --enable-abi-compat=no option, meaning they wish to forego binary compatibility in exchange for improved performance. These applications remain source portable to other ODP implementations.

Native odp_cpu_pause() for ARM processors

A native implementation of the odp_cpu_pause() API has been added for ARMv7 and ARMv8 processors.

New Microbenchmark

The odp_packet_bench microbenchmark application has been added to the test performance directory to allow implementations to measure and calibrate the performance of individual ODP packet APIs.

Ordered PktIO Test

The odp_pktio_ordered application has been added to the test performance directory to provide stress-testing of packet ordering features of ODP.

Documentation

In addition to expanded documentation related to the new packet reference APIs, a section on applicatin portability has been added that discusses the types of portability offered by ODP and the tradeoffs that application writers should consider when using ODP.

Bug Fixes

Bug 2806

The hello.c application can now run properly if Core 0 is not available (any available core will be used).

Bug 2827

Provide proper return code checking within _ishm.c

Bug 2829

Remove unused variables in iplookuptable helper routine.

Bug 2830

Avoid memory leaks on error paths in the cuckoo table helper functions.

Bug 2834

Fixes a race condition in shared memory allocation.

Bug 2842

Provide proper fallback for shared memory when hugepages are not available.

Bug 2862

Avoid null pointer dereference in the iplookuptable helper routine.

Bug 2865

Missing doxygen documentation for helper table functions are added.

Known Issues

Bug 2852

C++ programs fail to compile if they attempt to include the ODP helper APIs.

Bug 2895

The odp_crypto_operation() routine does not work with multi-segment packets.

OpenDataPlane (1.13.0.0)

New Features

APIs

ODP v1.13.0.0 represents the initial preview of the Tiger Moth release series and as such introduces new APIs and extensions that will be built on as this release matures.

Crypto Parameter Normalization

Many ODP APIs take parameter structs of type odp_xxx_param_t. The crypto APIs, for historical reasons, failed to follow this convention, using instead structs of type odp_crypto_params_t, etc. These pluralized names are now deprecated and their singular forms are introduced as the preferred names for these structs. The old names are retained (for now) to ease migration, however applications should convert to the new forms at their convenience as these deprecated forms will be removed in the final Tiger Moth release.

The changes introduced for this include:

  • odp_crypto_op_params_todp_crypto_op_param_t

  • odp_crypto_session_params_todp_crypto_session_param_t

Crypto Decouple Key Length from Algorithm Specification

To provide a more flexible programming for handling all possible key/digest/iv length combinations, the enums used for crypto specification are split to decouple lengths from algorithm names. The only exception is the SHA-2 family of hash routines, which have well-established naming conventions that incorporate digest lengths (SHA-224, SHA-256, etc.)

Changes included with this restructure include:

  • The odp_crypto_capability_t structure returned by the odp_crypto_capability() API contains streamlined odp_crypto_cipher_algos_t and odp_crypto_auth_algos_t substructures.

  • A new odp_crypto_cipher_capability() API is added to return detailed information about available cipher capabilities.

  • A new odp_crypto_auth_capability() API is added to return detailed information about available authentication capabilities.

odp_crypto_session_param_init() API

For completeness the odp_crypto_session_param_init() API is provided to enable platform-independent initialization of the odp_crypto_session_param_t structure used as input to odp_crypto_session_create().

Bitfield and Byteorder Cleanup

The ODP_BITFIELD_ORDER define is added to the ODP specification to parallel the existing ODP_BYTEFIELD_ORDER define. This will be set to the value ODP_BIG_ENDIAN_BITFIELD or ODP_LITTLE_ENDIAN_BITFIELD. This also addresses Bug 2402, however since fixing this required a small API change this was deferred until an API release boundary.

Improved Name Argument Definitions in odp_xxx_create() Functions

The use of name arguments to the various resource creation APIs has been standardized and documentation improved to clarify that unique names are not required and that these may be specified as NULL for anonymous resource creation. When non-unique resource names are used, it is unspecified which of these are returned by a corresponding lookup API.

Pool Parameters for Packet Pools

The odp_pool_param_t structure adds the new field max_len to be used in packet pools to advise the implementation of the maximum sized packet that the application will allocate with a single odp_packet_alloc() call. This may enable storage pool optimization.

Packet Clarifications

API documentation for odp_packet_concat() and odp_packet_copy_from_pkt() is clarified to specify that the source and destination packets supplied to these APIs should not refer to the same packet.

Packet Allocation Length Clarification

API documentation for odp_packet_alloc() is clarified to specify that the supplied length for requested packet allocation should be greater than zero.

Random API Changes

The APIs provided for working with random data have been revised. The third parameter to odp_random_data() is now of type odp_random_kind_t, which is an enum that is used to specify the quality of random data required. The kinds of random data defined are:

ODP_RANDOM_BASIC

No specific quality guarantees. This is assumed to be pseudo-random data generated by software where speed of generation is more important than the quality of the results.This is the lowest kind of random.

ODP_RANDOM_CRYPTO

Random data suitable for use in cryptographic operations.

ODP_RANDOM_TRUE

True random data generated from a hardware entropy source. This is the highest kind of random.

The odp_random_max_kind() API is provided that returns the highest kind of data available on this implementation. Requests for higher kinds than can be supplied will fail.

For testing purposes it is often desirable to generate "random" sequences that are repeatable. To support this use case, the odp_random_test_data() API is introduced. This always returns BASIC random data but uses a user-supplied 64-byte seed parameter that is update for each call and can be used to repeat the same sequence as needed.

Shared Memory Improvements

The odp_shm_reserve() API adds two new additional flags to support external memory.

  • ODP_SHM_SINGLE_VA guarantees that all ODP threads sharing this memory will see the block at the same virtual address regardless of whether threads are implemented as pthreads or processes and when fork() calls are made to create them.

  • ODP_SHM_EXPORT allows the memory block to be visible to other ODP instances. Other ODP instances can retrieve this block via the new odp_shm_import() API.

Classification Clarifications

The relationship between classification and queue configuration in the odp_pktin_queue_param_t structure is clarified to emphasize that classification subsumes queue-based I/O processing. This is an API documentation change only.

Helpers

New helper APIs are introduced for IP table lookup support for longest prefix matching as well as cuckoo hash tables. These are designed to provide analogs to functions available in DPDK to assist applications being ported to ODP.

Performance Improvements

The odp-linux reference implementation includes a number of improvements to make it more suitable for production use on platforms that rely on software implementations of key ODP APIs.

Ring-based Pool Implementation

Storage pools used for buffers and packets are now implemented via lockless ring structures that support burst mode read/writes to local caches for significantly improved multi-core scalability

Buffer/Packet Header Optimizations

The layout of the structs used to support buffers and packets has been optimized to reduce cache footprint and minimize path lengths in packet manipulation.

Ordered Queue Improvements

The implementation of ordered queues has been streamlined and made more scaleable in multi-core systems.

Packet Segmentation Improvements

The more advance segmentation capabilities present in the new ODP packet implementation are used to streamline the implementation of APIs like odp_packet_concat() and the head/tail extend/trunc APIs.

Bug Fixes

Bug 2405

A number of "todos" were removed from the packet validation test suite.

Bug 2472

The CPU affinity is now correctly read from the cpuset.

Bug 2496

The PktIO validation test no longer uses invalid MAC addresses.

Bug 2512

The TCP checksum is now properly calculated for generated packets.

Bug 2798

The odp-linux reference implementation makes use of the OpenSSL library to support the odp_random_xxx() APIs and some crypto operations. To support OpenSSL versions prior to 1.1.0, which are not thread safe, support is added for OpenSSL locking callbacks that use ODP ticketlocks to provide OpenSSL thread safety.

Known Issues

Bug 2812

Make check fails on a single core VM in the process mode helper test.

OpenDataPlane (1.12.0.0)

New Features

APIs

ODP v1.12.0.0 has no API changes from previous v1.11.0 Monarch LTS. Version is increased in current development release to make room for Monarch updates numbers.

Application Binary Interface (ABI) Support

Support is added to enable ODP applications to be binary compatible across different implementations of ODP sharing the same Instruction Set Architecture (ISA). This support introduces a new configure option:

no abi disable option

This is the default and specifies that the ODP library is to be built to support ABI compatibility mode. In this mode ODP APIs are never inlined. ABI compatibility ensures maximum application portability in cloud environments.

--disable-abi-compat

Specify this option to enable the inlining of ODP APIs. This may result in improved performance at the cost of ABI compatibility and is suitable for applications running in embedded environments.

Note that ODP applications retain source code portability between ODP implementations regardless of the ABI mode chosen. To move to a different ODP application running on a different ISA, code need simply be recompiled against that target ODP implementation.

SCTP Parsing Support

The ODP classifier adds support for recognizing Stream Control Transmission Protocol (SCTP) packets. The APIs for this were previously not implemented.

Packaging and Implementation Refinements

Remove dependency on Linux headers

ODP no longer has a dependency on Linux headers. This will help make the odp-linux reference implementation more easily portable to non-Linux environments.

Remove dependency on helpers

The odp-linux implementation has been made independent of the helper library to avoid circular dependency issues with packaging. Helper functions may use ODP APIs, however ODP implementations should not use helper functions.

Reorganization of test directory

The test directory has been reorganized to better support a unified approach to ODP component testing. API tests now live in test/common_plat/validation/api instead of the former test/validation. With this change performance and validation tests, as well as common and platform-specific tests can all be part of a unified test hierarchy.

The resulting test tree now looks like:

New test directory hierarchy
test
├── common_plat
│   ├── common
│   ├── m4
│   ├── miscellaneous
│   ├── performance
│   └── validation
│       └── api
│           ├── atomic
│           ├── barrier
│           ├── buffer
│           ├── classification
│           ├── cpumask
│           ├── crypto
│           ├── errno
│           ├── hash
│           ├── init
│           ├── lock
│           ├── packet
│           ├── pktio
│           ├── pool
│           ├── queue
│           ├── random
│           ├── scheduler
│           ├── shmem
│           ├── std_clib
│           ├── system
│           ├── thread
│           ├── time
│           ├── timer
│           └── traffic_mngr
├── linux-generic
│   ├── m4
│   ├── mmap_vlan_ins
│   ├── performance
│   ├── pktio_ipc
│   ├── ring
│   └── validation
│       └── api
│           ├── pktio
│           └── shmem
└── m4

Pools

The maximum number of pools that may be created in the odp-linux reference implementation has been raised from 16 to 64.

Upgrade to DPDK 16.07

The DPDK pktio support in odp-linux has been upgraded to work with DPDK 16.07. A number of miscellaneous fixes and performance improvements in this support are also present.

PktIO TAP Interface Classifier Support

Packet I/O interfaces operating in TAP mode now can feed packets to the ODP classifier the same as other pktio modes can do.

Performance Improvements

Burst-mode buffer allocation

The scheduler and pktio components have been reworked to use burst-mode buffer allocation/deallocation, yielding a measurable performance gain in almost all cases.

Burst-mode queue operations

ODP queues internally now attempt to use burst-mode enq/deq operations to accelerate performance where applicable.

Ring-based Scheduler Priority Queues

The ODP scheduler has been enhanced to use ring-based priority queues, resulting in significantly better scalability in many core environments.

GitHub Automation Support

ODP now supports the Travis framework needed to trigger CI automation in conjunction with GitHub. This support is considered experimental for now.

Examples

New l3fwd Example

A new example application is provided that illustrates use of ODP for simple Layer 3 forwarding across multiple interfaces.

Documentation

Pure API Documentation

ODP now generates "pure" (implementation independent) doxygen documentation in addition to the specific documentation for the odp-linux implementation. The pure version is applicable to any ODP implementation as it simply describes the ODP API specification. Implementation-specific versions of this documentation describe both the APIs as well as specifics concerning typedefs, enums, etc. This should help clarify what aspects of ODP are platform independent.

Clarify ODP Thread Definition

The definition of an ODP thread in odp-linux has been clarified to specify that for this implementation ODP threads are Linux pthreads or Linux processes. Currently threads as processes is considered experimental in odp-linux.

Bug Fixes

Numerous refinements have been incorporated to make validation tests more robust in API coverage as well as correcting corner cases in the implementation of many ODP APIs. Notable fixes include:

Bug 2316

ODP_TIMEOUT_INVALID is now defined consistently with other pool elements so the restriction on using Pool 0 as a timer pool is removed.

Bug 2310

The odp_packet_copy() API now correctly handles user areas between pools that are configured with different sized per-packet user area definitions.

Bug 2571

Corrects the implementation of AES GCM decryption in the ODP crypto API.

Known Issues

Bug 2309

The Timer validation test fails sporadically in environments with high core counts.

OpenDataPlane (1.11.0.0)

ODP v1.11.0 is the base tag and branch for the Monarch LTS (Long Term Support) release of ODP.

New Features

APIs

As part of the final Monarch LTS API set, a minor change to the odp_tm_egress_t struct was made to better reflect the capabilities of SoC platforms targeting Monarch support. This change adds the boolean egress_fcn_supported that indicates whether TM systems support an egress function. In addition, each TM system is now associated with a PktIO rather than a PktOUT queue. This struct is input to the odp_tm_create() API and output from the odp_tm_find() API.

Strict Priority (SP) Scheduler

Building on the modular scheduler framework introduced in v1.10.1.0, An alternate Strict Priority (SP) scheduler is now available. The SP scheduler is selected when ODP is configured with the --enable_schedule_sp option.

The SP scheduler is designed to favor low-latency processing of high priority work at the expense of throughput. This is considered experimental code at this point and should be treated as such by those wishing to use it.

Bug Fixes

Scheduler PktIO Cleanup

The scheduler now properly cleans up PktIOs operating in SCHEDULE mode following odp_pktio_close() calls. This resolves Bug 2301.

Chksum Routine C++ conformance

The odph_chksum() helper routine is now C++ conformant. This resolves Bug 2302.

User Area Copying

The odp_packet_copy() API now copies the packet user area as part of its processing and the packet validation test has been enhanced to verify that this is done properly. This resolves Bug 2310.

Use of Pool 0 as a Timeout Pool

The internal definition of ODP_TIMEOUT_INVALID was changed in odp-linux to enable the use of Pool 0 as a timeout pool. This resolves Bug 2316.

Packaging

libodphelper

To facilitate API tracking, libodphelper.so has been split out from libodp.so.

Helper dependencies

To avoid circular dependencies that cause issues when packaging ODP as a shared library, the odp-linux implementation no longer makes and use of the ODP helper library.

Performance

Performance enhancements in the odp-linux reference implementation have been made in PktIO routines and the packet classifier.

General Cleanup

General cleanup throughout has resulted in adding proper termination calls to ODP examples as well as miscellaneous Doxygen corrections. Additionally, the odp-linux code base has been cleaned up by removing "To Dos" as well as extraneous #includes for extra C headers and dependencies on linux headers for improved portability.

Validation

Queue Capability Test

The Queue validation test now properly exercises the odp_queue_capability() API and attempts to create the maximum reported number of supported queues.

Platform Tests

The platform-specific tests have been moved from platform/linux-generic/test to test/platform/linux-generc so that all tests are now found under a single test directory.

Examples

The ODP examples directory has been cleaned up and numerous corrections to existing examples have been made to add proper termination paths, etc. In addition, a new "hello world" example (odp_hello.c) has been added that illustrates the basic startup and shutdown steps needed for all ODP applications. As a simple stand-alone application is is also a good initial build tests for new ODP implementations.

Documentation

Pure API Guide

The Doxygen API guide has been split so that both a "pure" version that is platform-independent and retains unresolved ODP abstract types as well as an implementation-specific version (for odp-linux) that shows platform-specific typedef and enum values.

Implementation Guide Improvements

The ODP Implementer’s Guide has been enhanced with a section on implementation considerations, including issues surrounding the definition of Application Binary Interfaces (ABIs).

Known Issues

Bug 2309

The timer validation tests have seen occasional failures when run on systems containing a larger number of CPUs (typically 24 or more).

Bug 2386

ODP cannot be compiled using the -m32 option to generate 32-bit code on a 64-bit platform when using GCC 4.9 (the default GCC found in Ubuntu 15.04). This is due to a known bug in GCC 4.9 and is closed as a permanent restriction.

Bug 2402

Doxygen generates a warning for ODP_BIG_ENDIAN_BITFIELD when compiling on little endian systems. A similar warning for ODP_LITTLE_ENDIAN_BITFIELD is issued when compiling on big endian systems.

Bug 2444

The traffic manager cannot be used with two PktIOs simultaneously. That is, odp-linux does not properly support applications that create more than one TM system. This is a bug in the odp-linux implementation that will be corrected. It is not a limit of the architecture or other ODP implementations.

Bug 2449

Packet queue pool causes out-of-bounds accesses. This is closely related to Bug 2444 and reflects that odp-linux does not properly support multiple TM systems. This is scheduled for correction in the first maintenance update for this ODP implementation.

OpenDataPlane (1.10.1.0)

New Features

APIs

ODP v1.10.1.0 is a maintenance release and as such introduces no new APIs or other changes that might result in application source incompatibility with v1.10.0.0.

Modular Scheduler

The ODP scheduler included as part of the odp-linux reference implementation has been modularized to support a "pluggable" architecture that will permit the easy introduction of alternate scheduler implementations that permit both easy experimentation as well as alternate scheduling models that are tuned to specific application workloads (i.e., a strict priority scheduler for latency-sensitive workloads). Included in this release is the just the foundation and alternate schedulers are expected to be added in future ODP releases.

Packaging

Rename of linux-generic to odp-linux

While the platform name within the ODP git repository remains linux-generic, the package name of this reference implementation has been changed to odp-linux to better reflect that this is an implementation of ODP that is dependent only on the Linux kernel and is intended to be runnable on any Linux distribution. All ODP documentation has been changed to reflect this name change.

Helpers

Introduction of "agnostic" threading support

The current helper APIs for linux pthreads (odph_linux_pthread_create(), etc.) and processes (odph_linux_process_fork(), etc.) have been augmented with a new "agnostic" set of helper routines for general ODP thread management.

odph_odpthreads_create() and odph_odpthreads_join() are now the preferred helpers to use for creating and joining threads that may be implemented as either pthreads or Linux processes. Although currently ODP threads in Monarch are assumed to share the same (single) address space, this change is to permit evolution towards multi-address space (i.e., process oriented) threading models without requiring application and/or validation test changes.

The various ODP examples and validation tests that make use of threading have all been changed to new these new helper APIs.

Thread affinity APIs

As part of the new "agnostic" threading support, two additional helper APIs (odph_odpthread_getaffinity() and odph_odpthread_setaffinity()) have been added to permit easy management of thread affinity.

The CUnit framework used by the ODP validation suite has been changed to set the affinity of the initial thread to the first available control CPU. This avoids inconsistencies in a number of timing tests.

Bug Fixes

As a maintenance release, numerous bugs have been fixed in the odp-linux reference implementation in areas such as platform support (particularly on 32-bit systems), compiler support (clang), C++ compatibility, and other cleanup items. Highlights include:

32-bit platform support

The configure utility now correctly identifies and sets variables needed for proper compilation on 32-bit x86 systems. In addition the Traffic Manager now executes properly on 32-bit systems.

Timer improvements

The odp_timer_t abstract type is now strongly typed for consistency with other ODP types, and the odp_timer_to_u64() and odp_timer_pool_to_u64() APIs are now properly implemented. In addition, an issue with compiling the timer routines on 32-bit systems using clang was resolved.

Scheduler fairness

The default scheduler has been improved to avoid starvation and increase fairness when running with a limited number of threads or CPUs.

128-bit atomics

The -mcx16 compiler option is now properly identified and used to support 128-bit atomics on systems that support these operations. This leads to increased efficiency in the implementation of both timers as well as other atomic operations.

Use of hugepages

In systems that support multiple hugepage sizes, the default hugepage size is now reported properly by the odp_sys_huge_page_size() API.

Coverity and Valgrind issue cleanup

Numerous issues identified by the Coverity scan tool have been cleaned up and resolved, leading to improved robustness in the odp-linux reference implementation. In addition, several memory leaks identified by the valgrind tool have been fixed.

Validation test resource checks

The Traffic Manager validation test has certain sub-tests that require a minimum of 2 CPUs to operate properly. These tests are now skipped with a warning when run on uniprocessor systems.

Performance

Performance enhancements in the odp-linux reference implementation have been made in packet processing, classification, and pktio, leading to increased observed throughput and packet handling rates.

Examples

A new l2fwd_simple example has been added that does simple Layer 2 forwarding.

Documentation

Switch from AsciiDoc to AsciiDoctor

To get better formatting capabilities and future extensibility the ODP user documentation system has been switched from AsciiDoc to AsciiDoctor. There are a handful of notable changes as a result of this. Those submitting documentation patches should be aware of the following:

  • monospace font is now indicated by demarking the text with backticks (`) rather than plus signs (+).

  • Callouts are now indicated by C++-style double slash comments (// <1>) instead of C-style comments (/* <1> */). So, for example writing

[source,c]
----
...some great code deserving a callout // <1>
----
<1> This is an example of a callout

yields:

...some great code deserving a callout (1)
  1. This is an example of a callout

Note that this change only affects user documentation. Code documentation still uses Doxygen.

Image Cleanup

The images in the user guide have been trimmed to fit the page better. In addition, many new images have been added to better illustrate ODP API usage.

Additional User Documentation for Monarch APIs

The ODP User’s Guide has been updated to include new sections documenting the use of the full Monarch packet APIs as well as the Timer APIs. In addition, the crypto, classification, and traffic manager API documentation has been improved to cover all of the latest Monarch features, and an introductory section on overall ODP packet flow architecture has been added.

Helper Documentation

The ODP Helper library now has its own document that describes these additional functions of use to ODP applications and validation tests.

Other documentation

The Process Guide, FAQ, CONTRIBUTING, and CHANGELOG files have been reworked for the new AsciiDoctor structure, leading to a more uniform method of presenting ODP support documentation. In addition, an option for generating both a "pure" ODP API document that defines the APIs in their implementation-independent forms as well as their implementation-dependent forms for the odp-linux reference implementation is now available.

Known Issues

Bug 2301

The scheduler does not remove PKTIN queues following odp_pktio_close()

Bug 2309

The timer validation tests have seen occasional failures when run on systems containing a larger number of CPUs (typically 24 or more).

OpenDataPlane (1.10.0.0)

New Features

General

  • Move to capabilite structures: The classification, crypto, pktio, pool, queue, shared memory, and traffic manager areas all add new APIs to query implementation-specific capability limits for enhanced program portability.

Classification

  • Addtion of a structure that defines system level classification capability

  • Addition of range PMRs to complement the existing value PMRs to permit application to specifiy classification rules over a range of values.

Cryptography

  • Provides a way to get the available cipher and authentication algorithms.

Debug

  • Adds the ODP_STATIC_ASSERT() API to permit compile-time assertions.

Packet

  • Rename and add new packet copy and move functionality.

  • Add packet prefetch API for performance optimization.

  • New APIs to permit packet headroom and tailroom to be extended to additional segments or truncated as needed.

  • Align an area of packet data (e.g. IP/UDP headers) so that it is contiguous in memory and starts in a specified minimum alignment.

  • Add packet time stamp metadata support.

  • Add packet input index metadata support to allow application context to be anchored from the associated interface that received the packet.

  • Deprecation and removal of segment-level buffer APIs that are now superseded by additional packet-level manipulation APIs discussed above.

Packet io

  • Replaced config definition for maximum pktio entries with maximum packet IO index call

  • Added the classify_enable bit to the odp_pktin_queue_param_t that allows applications to explicity control which pktin queues are subject to full classifier support.

  • Addition of pktin configuration options to control packet timestamping and checksum validation processing.

  • Addition of pktout configuration options to control packet checksum offload processing.

  • Add the ability to query (via capabilties) whether a pktio interface supports operating in loopback mode and if so to enable/disable this mode of operation.

  • Round out the polling APIs be adding the ability to receive packets in poll mode from one or more pktin queues with timeout.

  • Add the ability to set the pktio interface index for ease of setting up application context areas associated with each pktio.

  • Add the ability to inqure the timestamp resolution associated with pktio interfaces that support packet timestamping.

Queue

  • Add a queue capability API to query system-wide queue capabilities and limits.

  • Extend queue context with an explicit data length parameter that serves as a performance hint for the amount of context to prefetch when the scheduler selects an event from a scheduled queue.

Scheduler

  • Add the ability to query information about a defined scheduler group

Shared Memory

  • Add a capability inquiry API to obtain implementation shared memory limits.

Traffic Manager

  • Add a capability inquiry API to obtain implementation traffic mngr limits.

  • Split TOS marking into two separate calls.

  • Add new APIs to support VLAN, ECN, and drop precedence marking.

  • Add destroy APIs for shapers, scheduler profiles, threshholds, WRED profiles, TM queues, and TM nodes for symmetry and completeness to allow applictions to terminate gracefully without resource leakage.

  • Add the ability to disconnect TM queues from their fanouts.

  • Add TM node contexts to permit applications to anchor user context areas to TM nodes.

  • Add the ability to query info about TM node fanins and TM queues.

  • Deprecate and remove the odp_tm_periodic_update() API

Utilities/Helpers

  • More complete and correct checksum implementation

Known Issues

OpenDataPlane (1.9.0.0)

New Features

  • A focus has been made on improving the packaging support of ODP for distributions such as Debian.

  • Improved creation of source tar ball and packaging scenarios moved to https://git.linaro.org/lng/odp-packaging.git

  • Renamed libodp to libodp-linux

  • Removed all internal functions from leaking through the libodp-linux

Known Issues

Time and Traffic Manager validation tests sometimes fail on heavy loaded systems. This behaviour depends on time accuracy measurements, which depends on system load.