Is that a review comment? Is there a problem with the rendering? On Tue, Nov 17, 2015 at 11:51 AM, Mike Holmes <[email protected]> wrote:
> This is the rendered doc > > http://people.linaro.org/~mike.holmes/output/users-guide.html > > On 17 November 2015 at 12:15, Bill Fischofer <[email protected]> > wrote: > >> Add a basic ODP overview to the User's Guide, providing >> an overview of ODP concepts, components, etc. Included >> are a number of diagrams covering component structure >> as well as packet RX, event scheduling, and packet TX >> processing. >> >> Signed-off-by: Bill Fischofer <[email protected]> >> --- >> doc/users-guide/users-guide.adoc | 400 >> ++++++++++++++++++++++++++++++++++++++- >> 1 file changed, 398 insertions(+), 2 deletions(-) >> >> diff --git a/doc/users-guide/users-guide.adoc >> b/doc/users-guide/users-guide.adoc >> index 7f70046..cf77fa0 100644 >> --- a/doc/users-guide/users-guide.adoc >> +++ b/doc/users-guide/users-guide.adoc >> @@ -8,7 +8,7 @@ OpenDataPlane (ODP) Users-Guide >> Abstract >> -------- >> This document is intended to guide a new ODP application developer. >> -Further details about ODP may be found at then http://opendataplane.org[ODP] >> home page. >> +Further details about ODP may be found at the http://opendataplane.org[ODP] >> home page. >> >> .Overview of a system running ODP applications >> image::../images/overview.png[align="center"] >> @@ -16,6 +16,403 @@ image::../images/overview.png[align="center"] >> ODP is an API specification that allows many implementations to provide >> platform independence, automatic hardware acceleration and CPU scaling to >> high performance networking applications. >> This document describes how to write an application that can >> successfully take advantage of the API. >> >> +:numbered: >> +== Introduction == >> +.OpenDataPlane Components >> +image::../images/odp_components.png[align="center"] >> + >> +.The ODP API Specification >> +ODP consists of three separate but related component parts. First, ODP >> is an >> +abstract API specification that describes a functional model for >> +data plane applications. This specification covers many common data plane >> +application programming needs, such as the ability to receive, >> manipulate, and >> +transmit packet data, without specifying how these functions are >> performed. This >> +is quite intentional. It is precisely because ODP APIs do not have a >> preferred >> +embodiment that they permit innovation in how these functions can >> +be realized on various platforms that offer implementations of ODP. To >> achieve >> +this goal, ODP APIs are described using abstract data types whose >> definition >> +is left up to the ODP implementer. For example, in ODP packets are >> referenced >> +by abstract handles of type +odp_packet_t+, and packet-related APIs take >> +arguments of this type. What an +odp_packet_t+ actually is is not part >> of the >> +ODP API specification--that is the responsibility of each ODP >> implementation. >> + >> +.Summary: ODP API attributes: >> +* Open Source, open contribution, BSD-3 licensed. >> +* Vendor and platform neutral. >> +* Application-centric. Covers functional needs of data plane >> applications. >> +* Ensures portability by specifying the functional behavior of ODP. >> +* Defined jointly and openly by application writers and platform >> implementers. >> +* Archiected to be implementable on a wide range of platforms efficiently >> +* Sponsored, governed, and maintained by the Linaro Networking Group >> (LNG) >> + >> +.ODP Implementations >> +Second, ODP consists of multiple implementations of this API >> specification, >> +each tailored to a specific target platform. ODP implementations >> determine >> +how each ODP abstract type is represented on that platform and how each >> ODP >> +API is realized. On some platforms, ODP APIs will >> +be realized using specialized instructions that accelerate the functional >> +behavior specified by the API. On others, hardware co-processing engines >> may >> +completely offload the API so that again it can be performed with little >> or no >> +involvement by a CPU. In all cases, the application sees the same >> +functional behavior independent of how a given platform has chosen to >> realize >> +it. By allowing each platform the freedom to determine how best to >> realize each >> +API's specified functional behavior in an optimal manner, ODP permits >> +applications written to its APIs to take full advantage of the unique >> +capabilities of each platform without the application programmer needing >> to >> +have specialist knowledge of that platform or to be concerned with how >> best >> +to tune the application to a particular platform. This latter >> consideration is >> +particularly important in Network Function Virtualization (NFV) >> environments >> +where the application will run on a target platform chosen by someone >> else. >> + >> +.Summary: ODP Implementation Characteristics >> +* One size does not fit all--supporting multiple implementations allows >> ODP >> +to adapt to widely differing internals among platforms. >> +* Anyone can create an ODP implementation tailored to their platform >> +* Distribution and mainteinance of each implementation is as owner wishes >> + - Open source or closed source as business needs determine >> + - Have independent release cycles and service streams >> +* Allows HW and SW innovation in how ODP APIs are implemented on each >> platform. >> + >> +.Reference Implementations >> +To make it easy to get started with implementing ODP on a new platform, >> ODP >> +supplies a number of _reference implementations_ that can serve as a >> +starting point. The two primary references implementations supplied by >> ODP are >> +*odp-linux* and *odp-dpdk* >> + >> +.odp-linux >> +The *odp-linux* reference implementation is a pure SW implementation of >> the >> +ODP API that relies only on the Linux programming API. As a functional >> model >> +for ODP, it enables ODP to be bootstrapped easily to any platform that >> +supports a Linux kernel. >> + >> +.odp-dpdk >> +The *odp-dpdk* reference implementation is a pure SW implementation of >> the >> +ODP API that uses http://dpdk.org[DPDK] as a SW accelerator. In >> particular, >> +*odp-dpdk* offers superior I/O performance for systems that use NICs, >> allowing >> +ODP applications to take immediate full advantage of the various NIC >> device >> +drivers supported by DPDK. >> + >> +.Summary: ODP Reference Implementations >> +* Open source, open contribution, BSD-3 licensed. >> +* Provide easy bootstrapping of ODP onto new platforms >> +* Implementers free to borrow or tailor code as needed for their platform >> +* Implementers retain full control over their implementations whether or >> not >> +they are derived from a reference implementation. >> + >> +.ODP Validation Test Suite >> +Third, to enure consistency between different ODP implementations, ODP >> +consists of a validation suite that verifies that any given >> implementation of >> +ODP faithfully provides the specified functional behavior of each ODP >> API. >> +As a separate open source component, the validation suite may be used by >> +application writers, system integrators, and platform providers alike to >> +confirm that any purported implementation of ODP does indeed conform to >> the >> +ODP API specification. >> + >> +.Summary: ODP Validation Test Suite >> +* Synchronized with ODP API specification >> +* Maintained and distributed by LNG >> +* Open source, open contribution, BSD-3 licensed. >> +* Key to ensuring application portability across all ODP implementations >> +* Tests that ODP implementations conform to the specified functional >> behavior >> +of ODP APIs. >> +* Can be run at any time by users and vendors to validat implementations >> +od ODP. >> + >> +=== ODP API Specification Versioning === >> +As an evolving standard, the ODP API specification is released under an >> +incrementing version number, and corresponding implementations of ODP, >> as well >> +as the validation suite that verifies API conformance, are linked to this >> +version number. ODP versions are specified using a stanard three-level >> +number (major.minor.fixlevel) that are incremented according to the >> degree of >> +change the level represents. Increments to the fixlevel represent >> clarification >> +of the specification or other minor changes that do not affect either the >> +syntax or semantics of the specification. Such changes in the API >> specification >> +are expected to be rare. Increments to the minor level >> +represent the introduction of new APIs or functional capabilities, or >> changes >> +to he specified syntax or functional behavior of APIs and thus may >> require >> +application source code changes. Such changes are well documented in the >> +release notes for each revision of the specification. Finally, >> increments to >> +the major level represent significant structural changes that most likely >> +require some level of application source code change, again as >> documented in >> +the release notes for that version. >> + >> +=== ODP Implementation Versioning === >> +ODP implementations are free to use whatever release naming/numbering >> +conventions they wish, as long as it is clear what level of the ODP API >> a given >> +release implements. A recommended convention is to use the same three >> level >> +numbering scheme where the major and minor numbers correspond to the ODP >> API >> +level and the fixlevel represents an implementation-defined service level >> +associated with that API level implementation. The LNG-supplied ODP >> reference >> +implementations follow this convention. >> + >> +=== ODP Validation Test Suite Versioning === >> +The ODP validation test suite follows these same naming conventions. The >> major >> +and minor release numbers correspond to the ODP API level that the suite >> +validates and the fixlevel represents the service level of the validation >> +suite itself for that API level. >> + >> +=== ODP Design Goals === >> +ODP has three primary goals that follow from its component structure. >> The first >> +is application portability across a wide range of platforms. These >> platforms >> +differ in terms of processor instruction set architecture, number and >> types of >> +application processing cores, memory oranization, as well as the number >> and >> +type of platform specific hardware acceleration and offload features that >> +are available. ODP applications can move from one conforming >> implementation >> +to another with at most a recompile. >> + >> +Second, ODP is designed to permit data plane applications to avail >> themselves >> +of platform-specific features, including specialized hardware >> accelerators, >> +without specialized programming. This is achieved by separating the API >> +specification from their implementation on individual platforms. Since >> each >> +platform implements each ODP API in a manner optimal to that platform, >> +applications automatically gain the benefit of such optimizations >> without the >> +need for explicit programming. >> + >> +Third, ODP is designed to allow applications to scale out automatically >> to >> +support many core architectures. This is done using an event based >> programming >> +model that permits applications to be written to be independent of the >> number >> +of processing cores that are available to realize application function. >> The >> +result is that an application written to this model does not require >> redesign >> +as it scales from 4, to 40, to 400 cores. >> + >> +== Organization of this Document == >> +This document is organized into several sections. The first presents a >> high >> +level overview of the ODP API component areas and their associated >> abstract >> +data types. This section introduces ODP APIs at a conceptual level. >> +The second provides a tutorial on the programming model(s) >> +supported by ODP, paying particular attention to the event model as this >> +represents the preferred structure for most ODP applications. This >> section >> +builds on the concepts introduced in the first section and shows how ODP >> +applications are structured to best realize the three ODP design goals >> +mentioned earlier. The third section provides a more detailed overview of >> +the major ODP API components and is designed to serve as a companion to >> the >> +full reference specification for each API. The latter is intended to be >> used >> +by ODP application programmers, as well as implementers, to understand >> the >> +precise syntax and semantics of each API. >> + >> +== ODP API Concepts == >> +ODP programs are built around several conceptual structures that every >> +appliation programmer needs to be familiar with to use ODP effectively. >> The >> +main ODP concepts are: >> +Thread, Event, Queue, Pool, Shared Memory, Buffer, Packet, PktIO, Timer, >> +and Synchronizer. >> + >> +=== Thread === >> +The thread is the fundamental programming unit in ODP. ODP applications >> are >> +organized into a collection of threads that perform the work that the >> +application is designed to do. ODP threads may or may not share memory >> with >> +other threads--that is up to the implementation. Threads come in two >> "flavors": >> +control and worker, that are represented by the abstract type >> ++odp_thread_type_t+. >> + >> +A control thread is a supervisory thread that organizes >> +the operation of worker threads. Worker threads, by contrast, exist to >> +perform the main processing logic of the application and employ a run to >> +completion model. Worker threads, in particular, are intended to operate >> on >> +dedicated processing cores, especially in many core proessing >> environments, >> +however a given implementation may multitask multiple threads on a single >> +core if desired (typically on smaller and lower performance target >> +environments). >> + >> +In addition to thread types, threads have associated _attributes_ such as >> +_thread mask_ and _scheduler group_ that determine where they can run and >> +the type of work that they can handle. These will be discussed in greater >> +detail later. >> + >> +=== Event === >> +Events are what threads process to perform their work. Events can >> represent >> +new work, such as the arrival of a packet that needs to be processed, or >> they >> +can represent the completion of requests that have executed >> asynchronously. >> +Events can also represent notifications of the passage of time, or of >> status >> +changes in various components of interest to the application. Events >> have an >> +event type that describes what it represents. Threads can create new >> events >> +or consume events processed by them, or they can perform some processing >> on >> +an event and then pass it along to another component for further >> processing. >> +References to events are via handles of abstract type +odp_event_t+. Cast >> +functions are provided to convert these into specific handles of the >> +appropriate type represented by the event. >> + >> +=== Queue === >> +A queue is a message passing channel that holds events. Events can be >> +added to a queue via enqueue operations or removed from a queue via >> dequeue >> +operations. The endpoints of a queue will vary depending on how it is >> used. >> +Queues come in two major types: polled and scheduled, which will be >> +discussed in more detail when the event model is introduced. Queues may >> also >> +have an associated context, which represents a persistent state for all >> +events that make use of it. These states are what permit threads to >> perform >> +stateful processing on events as well as stateless processing. >> + >> +Queues are represented by handles of abstract type +odp_queue_t+. >> + >> +=== Pool === >> +A pool is a shared memory area from which elements may be drawn. Pools >> +represent the backing store for events, among other things. Pools are >> +typically created and destroyed by the application during initialization >> and >> +termination, respectively, and then used during processing. Pools may be >> +used by ODP components exclusively, by applications exclusively, or their >> +use may be shared between the two. Pools have an associated type that >> +characterizes the elements that they contain. The two most important >> pool types >> +are Buffer and Packet. >> + >> +Pools are represented by handles of abstract type +odp_pool_t+. >> + >> +=== Shared Memory === >> +Shared memory represents raw blocks of storage that are sharable between >> +threads. They are the building blocks of pools but can be used directly >> by >> +ODP applications if desired. >> + >> +Shared memory is represented by handles of abstract type +odp_shm_t+. >> + >> +=== Buffer === >> +A buffer is a fixed sized block of shared storage that is used by ODP >> +components and/or applications to realize their function. Buffers contain >> +zero or more bytes of application data as well as system maintained >> +metadata that provide information about the buffer, such as its size or >> the >> +pool it was allocated from. Metadata is an important ODP concept because >> it >> +allows for arbitrary amounts of side information to be associated with an >> +ODP object. Most ODP objects have assocaited metadata and this metadata >> is >> +manipulated via accessor functions that act as getters and setters for >> +this information. Getter acces functions permit an application to read >> +a metadata item, while setter access functions permit an application to >> write >> +a metadata item. Note that some metadata is inherently read only and thus >> +no setter is provided to manipulate it. When object have multiple >> metadata >> +items, each has its own associated getter and/or setter access function >> to >> +inspect or manipulate it. >> + >> +Buffers are represened by handles of abstract type +odp_buffer_t+. >> + >> +=== Packet === >> +Packets are received and transmitted via I/O interfaces and represent >> +the basic data that data plane applications manipulate. >> +Packets are drawn from pools of type +ODP_POOL_PACKET+. >> +Unlike buffers, which are simple objects, >> +ODP packets have a rich set of semantics that permit their inspection >> +and manipulation in complex ways to be described later. Packets also >> support >> +a rich set of metadata as well as user metadata. User metadata permits >> +applications to associate an application-determined amount of side >> information >> +with each packet for its own use. >> + >> +Packets are represented by handles of abstract type +odp_packet_t+. >> + >> +=== PktIO === >> +PktIO is how ODP represents I/O interfaces. A pktio object is a logical >> +port capable of receiving and/or transmitting packets. This may be >> directly >> +supported by the underlying platform as an integrated feature, >> +or may represent a device attached via a PCIE or other bus. >> + >> +PktIOs are represented by handles of abstract type +odp_pktio_t+. >> + >> +=== Timer === >> +Timers are how ODP applications measure and respond to the passage of >> time. >> +Timers are drawn from specialized pools called timer pools that have >> their >> +own abstract type (+odp_timer_pool_t+). Applications may have many timers >> +active at the same time and can set them to use either relative or >> absolute >> +time. When timers expire they create events of type +odp_timeout_t+, >> which >> +serve as notifications of timer expiration. >> + >> +=== Synchronizer === >> +Multiple threads operating in parallel typically require various >> +synchronization services to permit them to operate in a reliable and >> +coordinated manner. ODP provides a rich set of locks, barriers, and >> similar >> +synchronization primitives, as well as abstract types for representing >> various >> +types of atomic variables. The ODP event model also makes use of queues >> to >> +avoid the need for explicit locking in many cases. This will be discussed >> +in the next section. >> + >> +== ODP Components == >> +Building on ODP concepts, ODP offers several components that relate to >> the >> +flow of work through an ODP application. These include the Classifier, >> +Scheduler, and Traffic Manager. These components relate to the three >> +main stages of packet processing: Receive, Process, and Transmit. >> + >> +=== Classifier === >> +The *Classifier* provides a suite of APIs that control packet receive >> (RX) >> +processing. >> + >> +.ODP Receive Processing with Classifier >> +image::../images/odp_rx_processing.png[align="center"] >> + >> +The classifier provides two logically related services: >> +[horizontal] >> +Packet parsing:: Verifying and extracting structural information from a >> +received packet. >> + >> +Packet classification:: Applying *Pattern Matching Rules (PMRs)* to the >> +parsed results to assign an incoming packet to a *Class of Service >> (CoS)*. >> + >> +Combined, these permit incoming packets to be sorted into *flows*, which >> are >> +logically related sequences of packets that share common processing >> +requirements. While many data plane applications perform stateless packet >> +processing (_e.g.,_ for simple forwarding) others perform stateful packet >> +processing. Flows anchor state information relating to these groups of >> +packets. >> + >> +A CoS determines two variables for packets belonging to a flow: >> +[list] >> +* The pool that they will be stored in on receipt >> +* The queue that they will be added to for processing >> + >> +The PMRs supported by ODP permit flow determination based on >> combinations of >> +packet field values (tuples). The main advantage of classification is >> that on >> +many platforms these functions are performed in hardware, meaning that >> +classification occurs at line rate as packets are being received without >> +any explicit processing by the ODP application. >> + >> +Note that the use of the classifier is optional. Applications may >> directly >> +receive packets from a corresponding PktIO input queue via direct polling >> +if they choose. >> + >> +=== Scheduler === >> +The *Scheduler* provides a suite of APIs that control scalabable event >> +processing. >> + >> +.ODP Scheduler and Event Processing >> +image::../images/odp_scheduling.png[align="center"] >> + >> +The Scheduler is responsible for selecting and dispatching one or more >> events >> +to a requesting thread. Event selection is based on several factors >> involving >> +both the queues containing schedulable events and the thread making an >> ++odp_schedule()+ or +odp_schedule_multi()+ call. >> + >> +ODP queues have a _scheduling priority_ that determines how urgently >> events >> +on them should be processed relative to events contained in other queues. >> +Queues also have a _scheduler group id_ associated with them that must >> match >> +the associated scheduler group _thread mask_ of the thread calling the >> +scheduler. This permits events to be grouped for processing into classes >> and >> +have threads that are dedicated to processing events from specified >> classes. >> +Threads can join and leave scheduler groups dynamically, permitting easy >> +application response to increases in demand. >> + >> +When a thread receives an event from the scheduler, it in turn can invoke >> +other processing engines via ODP APIs (_e.g.,_ crypto processing) that >> +can operate asynchronously. When such processing is complete, the result >> is >> +that a *completion event* is added to a schedulable queue where it can be >> +scheduled back to a thread to continue processing with the results of the >> +requested asynchronous operation. >> + >> +Threads themselves can enqueue events to queues for downstream processing >> +by other threads, permitting flexibility in how applicaitions structure >> +themselves to maximize concurrency. >> + >> +=== Traffic Manager === >> +The *Traffic Manager* provides a suite of APIs that control traffic >> shaping and >> +Quality of Service (QoS) processing for packet output. >> + >> +.ODP Transmit processing with Traffic Manager >> +image::../images/odp_traffic_manager.png[align="center"] >> + >> +The final stage of packet processing is to transmit it. Here, >> applications have >> +several choices. As with RX processing, applications may send packets >> +directly to PktIO TX queues for direct transmission. Often, however, >> +applications need to perform traffic shaping and related >> +*Quality of Service (QoS)* processing on the packets comprising a flow >> as part >> +of transmit processing. To handle this need, ODP provides a suite of >> +*Traffic Manager* APIs that permit programmatic establishment of >> arbiters, >> +shapers, etc. that control output packet processing to achieve desired >> QoS >> +goals. Again, the advantage here is that on many platforms traffic >> management >> +functions are implemented in hardware, permitting transparent offload of >> +this work. >> + >> Glossary >> -------- >> [glossary] >> @@ -28,7 +425,6 @@ odp_thread:: >> event:: >> An event is a notification that can be placed in a queue. >> >> -:numbered: >> The include structure >> --------------------- >> Applications only include the 'include/odp.h file which includes the >> 'platform/<implementation name>/include/plat' files to provide a complete >> definition of the API on that platform. >> -- >> 2.1.4 >> >> _______________________________________________ >> lng-odp mailing list >> [email protected] >> https://lists.linaro.org/mailman/listinfo/lng-odp >> > > > > -- > Mike Holmes > Technical Manager - Linaro Networking Group > Linaro.org <http://www.linaro.org/> *│ *Open source software for ARM SoCs > > >
_______________________________________________ lng-odp mailing list [email protected] https://lists.linaro.org/mailman/listinfo/lng-odp
