Attached is an initial draft of the Packet I/O design document.
(The document attached is provided under the terms of agreement LES-LTM-21309)
--
Stuart.
/* Copyright (c) 2014, Linaro
* All rights reserved
*
* SPDX-License-Identifier: BSD-3-Clause
*/
/**
@page pktio ODP Design - Packet I/O
For the implementation of the ODP Packet I/O API please see odp_packet_io.h
@tableofcontents
@section intro Introduction
This document describes the Packet I/O (pktio) APIs supported by ODP. These
APIs provide a high level abstraction of a single packet I/O interface,
allowing an ODP application to configure, send and receive packets over an
interface without having to concern itself with how the implementation performs
those operations. The packet I/O implementation for a given interface may be
performed entirely in hardware, entirely in software or a combination of the
two.
A pktio interface may represent a single physical, virtual or logical interface
(for example, a loopback interface). An ODP implementation on a single platform
may support multiple different interface types and export some or all of them
all via the pktio API.
An application would not typically receive packets directly from the pktio
interface, but would instead configure the system such that packets received
on the interface are classified and delivered to a predefined input queue, or
input queue group, depending on application requirements. The application then
receives packets from one of the input queues by invoking the scheduler.
Packets received from a pktio interface, either directly or indirectly, have
already been through input parsing and classification based on the rules
established when the interface was configured.
Similarly for transmission, a pktio interface has an output queue associated
with it. Packets are transmitted by enqueueing them to the output queue
associated with the pktio device over which the packet is to be sent, the
implementation is then responsible for taking the packet from the output queue
and effecting its transmission.
@section func Functional Description
The pktio APIs define the following handles to be used to access a pktio
interface;
- @c odp_pktio_t one per interface, all configuration of the interface is
performed using a handle of this type. There can be only one handle of this
type per interface at any time.
- @c odp_pktio_worker_t one per interface for each worker thread, used for I/O
and read only access to pktio configuration.
@subsection initialisation_sequence Initialisation sequence
The following section walks through a simplified example of an initialisation
sequence an ODP application may go through to use a pktio device.
1. Call @c odp_global_init(), if necessary passing platform specific information
required to configure the pktio interfaces.
2. Call @c odp_cos_create() to create a CoS.
3. Call @c odp_buffer_pool_create() to create pool of buffers of type
@c ODP_BUFFER_TYPE_PACKET and and call @c odp_cos_set_pool() to associate it
with the CoS, so that packets matching this CoS will be allocated from this
pool.
4. Call @c odp_queue_create() to create a queue of type ODP_QUEUE_TYPE_PKTIN
and call @c odp_pktio_set_default_cos() to associate it with the CoS so that
matching packets matching this CoS will be delivered to this queue.
5. Call @c odp_pktio_open() passing the above CoS as the default CoS for the
interface. As there are no L2/L3/PMR rules paired with the pktio interface,
all received packets will be assigned the default class of service.
6. Start calling @c odp_schedule() from each of the worker tasks to begin
receiving packets.
7. For each received packet, call @c odp_packet_outq() to determine which
output queue is to be used for transmission based on which interface the
packet was received on.
8. Call @c odp_queue_enq() to queue the packet for transmission on the selected
output queue.
Steps 1 to 5 would typically be performed at application startup in the main
thread. The worker tasks at step 7 could be active prior to any of the previous
steps, but packets only start being delivered after the association between the
pktio interface and the input queue is made during the odp_pktio_open() call.
Further examples showing different ways an application can use the API are
found in the \ref examples section later in the document.
@subsection ordering Ordering and Atomicity
There are two methods of receiving or sending packets via a pktio device,
either working directly with the pktio device via odp_pktio_recv/send or
indirectly via queues and the scheduler. The application writer must select
the method that best suits their requirements, using both methods for a single
interface is not supported.
In ODP, ordering and atomicity are functions of the scheduler, and in many cases
obtaining packets via the scheduler is how the best performance will be
achieved,
particularly on platforms that make use of a hardware accelerated scheduler.
Directly invoking send/recv on the pktio interface avoids the scheduler and
therefore the application is given no guarantees of ordering or atomicity.
There are however some cases where this can offer a performance improvement, for
example if the application maintains order by some other means such as only
accessing and interface from a single core, as it avoids the need to invoke the
scheduler and enqueue/dequeue packets.
@subsection ifacetypes Supporting multiple interface types
Some platforms have a number of different types of pktio interface, or at least
a number of different configuration options for a single type of interface. In
order for the ODP API to remain portable these differences must not require the
application to take platform specific code paths. The general mechanism for
coping with this is to assign a unique identifier to each interface (a character
string), this identifier is essentially opaque to the API but it used by the
implementation to determine which underlying interface is being accessed and
therefore the mechanisms used to work with it.
In addition to this the application can provide platform specific
initialisation data to odp_global_init() which can be used to configure
non-portable properties of the interface implementation.
@section apidef API Definitions
@subsection datatypescfg Data Types
The following data types are referenced in the API descriptions described
below. The names are part of the ODP API and MUST be present in any conforming
implementation, however the type values shown here are illustrative and
implementations SHOULD either use these or substitute their own type values
that are appropriate to the underlying platform.
@code
/* ODP packet IO handle, one per interface */
typedef uint32_t odp_pktio_t;
/* Invalid packet IO handle */
#define ODP_PKTIO_INVALID 0
/* ODP packet IO worker handle, one per interface for each worker thread */
typedef uint32_t odp_pktio_worker_t;
/* Invalid packet IO worker handle */
#define ODP_PKTIO_WORKER_INVALID 0
@endcode
@subsection odp_pktio_open odp_pktio_open
@code
/*
* Open an ODP packet IO instance.
*
* @param iface Packet IO interface name.
* @param default_cos Class-of-service to be assigned to packets received on
* this instance if no other class-of-service is matched.
*
* @return ODP packet IO handle or ODP_PKTIO_INVALID on error.
*/
odp_pktio_t odp_pktio_open(const char *iface, odp_cos_t default_cos);
@endcode
Open the specified interface and return an @c odp_pktio_t handle to be used to
refer to this pktio instance in subsequent API calls. A return value of
@c ODP_PKTIO_INVALID indicates failure and the caller should inspect errno to
determine the cause.
This routine would typically be called during application startup for each
interface the application intends to use. Each interface can be opened only
once.
The interface name is a NULL terminated string containing a human readable name
for the interface, used by the implementation to identify the underlying pktio
interface.
The default class-of-service is used to define how received packets should be
dealt with if they don't match any other class-of-service. A value of
@c ODP_COS_INVALID causes all such packets to be discarded. Immediately
following the odp_pktio_open() call all incoming packets will match the default
class-of-service until additional header based rules (either PMR or L2/L3
priority) have been configured. The default class-of-service can be modified at
any time by calling odp_pktio_set_default_cos().
@subsection odp_pktio_close odp_pktio_close
@code
/*
* Close an ODP packet IO instance.
*
* @param id ODP packet IO handle.
*
* @return 0 on success, -1 on error.
*/
int odp_pktio_close(odp_pktio_t id);
@endcode
Close a pktio instance and free any internal resources associated with it. It
is the responsibility of the application to ensure that any resources that have
been associated with this instance are freed, for example the default CoS.
@subsection odp_pktio_outq_getdef odp_pktio_outq_getdef
@code
/*
* Query default output queue associated with a pktio handle.
*
* @param id ODP packet IO handle.
*
* @return Default output queue or ODP_QUEUE_INVALID on error.
*/
odp_queue_t odp_pktio_outq_getdef(odp_pktio_t id);
@endcode
This routine returns the output queue associated with a pktio handle, packets
enqueued to this queue are queued for tranmission over the interface. The
output queue is assigned by the implementation during the open call.
@subsection odp_pktio_set_default_cos odp_pktio_set_default_cos
@code
/*
* Setup default class-of-service for a packet IO interface.
*
* @param id ODP packet IO handle.
* @param cos class-of-service set to assign to all packets arriving at the
* pktio interface, unless overridden by subsequent header-based
* filters.
*
* @return 0 on success, -1 on error.
*/
int odp_pktio_set_default_cos(odp_pktio_t id, odp_cos_t cos);
@endcode
@subsection odp_pktio_set_error_cos odp_pktio_set_error_cos
@code
/*
* Setup error class-of-service for a packet IO interface.
*
* @param id ODP packet IO handle.
* @param cos class-of-service set to all packets arriving the specified
* interface that contain an error.
*
* @return 0 on success, -1 on error.
*/
int odp_pktio_set_error_cos(odp_pktio_t id, odp_cos_t cos);
@endcode
This OPTIONAL routine assigns a class-of-service, which may be used to discard
such packets or deliver them to the application via a prescribed queue for
further processing. The specific errors types include L2 FCS and optionally
L3/L4 checksum errors, malformed headers etc. depending on platform
capabilities.
@subsection odp_pktio_set_input odp_pktio_set_input
@code
/*
* Store packet input handle into packet
*
* @param pkt ODP packet buffer handle
* @param id ODP packet IO handle
*
* @return
*/
void odp_pktio_set_input(odp_packet_t pkt, odp_pktio_t id);
@endcode
@subsection odp_pktio_get_input odp_pktio_get_input
@code
/*
* Get stored packet input handle from packet
*
* @param pkt ODP packet buffer handle
*
* @return Packet IO handle
*/
odp_pktio_t odp_pktio_get_input(odp_packet_t pkt);
@endcode
@subsection odp_pktio_set_mac_addr odp_pktio_set_mac_addr
@code
/*
* Set the default MAC address of a packet IO interface.
*
* @param id ODP packet IO handle.
* @param mac_addr MAC address to be assigned to the interface.
* @param addr_size Size of the address in bytes.
*
* @return 0 on success, -1 on error.
*/
int odp_pktio_set_mac_addr(odp_pktio_t id, unsigned char *mac_addr,
size_t addr_size);
@endcode
@subsection odp_pktio_mac_addr odp_pktio_mac_addr
@code
/*
* Get the default MAC address of a packet IO interface.
*
* @param id ODP packet IO handle.
* @param[out] mac_addr Storage for MAC address of the packet IO interface.
* @param[out] addr_size Size of the MAC address.
*
* @return 0 on success, -1 on error.
*/
int odp_pktio_mac_addr(odp_pktio_t id, unsigned char *mac_addr,
size_t *addr_size);
@endcode
@subsection odp_pktio_set_mtu odp_pktio_set_mtu
@code
/*
* Configure the MTU for a packet IO interface.
*
* @param id ODP packet IO handle.
* @param mtu The MTU to be applied.
*
* @return 0 on success, -1 on error.
*/
int odp_pktio_set_mtu(odp_pktio_t id, uint16_t mtu);
@endcode
@subsection odp_pktio_mtu odp_pktio_mtu
@code
/*
* Get the currently configured MTU of a packet IO interface.
*
* @param id ODP packet IO handle.
* @param[out] mtu Pointer to location in which to store the MTU value.
*
* @return 0 on success, -1 on error.
*/
int odp_pktio_mtu(odp_pktio_t id, uint16_t *mtu);
@endcode
@subsection odp_pktio_promisc_enable odp_pktio_promisc_enable
@code
/*
* Enable promiscuous mode on a packet IO interface.
*
* @param id ODP packet IO handle.
*
* @return 0 on success, -1 on failure.
*/
int odp_pktio_promisc_enable(odp_pktio_t id);
@endcode
@subsection odp_pktio_promisc_disable odp_pktio_promisc_disable
@code
/*
* Disable promiscuous mode on a packet IO interface.
*
* @param id ODP packet IO handle.
*
* @return 0 on success, -1 on failure.
*/
int odp_pktio_promisc_disable(odp_pktio_t id);
@endcode
@subsection odp_pktio_promisc odp_pktio_promisc
@code
/*
* Determine if promiscuous mode is enabled for a packet IO interface.
*
* @param id ODP packet IO handle.
*
* @return 0 if promiscuous mode is disabled, 1 if enabled.
*/
int odp_pktio_promisc(odp_pktio_t id);
@endcode
@section apidefworkers API Definitions - Workers
@subsection odp_pktio_worker_open odp_pktio_worker_open
@code
/*
* Open an ODP packet IO worker instance.
*
* @param iface Packet IO interface name.
*
* @return ODP packet IO worker handle or ODP_PKTIO_WORKER_INVALID on error.
*/
odp_pktio_worker_t odp_pktio_worker_open(const char *iface);
@endcode
Open a handle to be used when direct access to a pktio instance is required
from a worker thread. The interface must have already been opened using
@c odp_pktio_open.
@subsection odp_pktio_worker_close odp_pktio_worker_close
@code
/*
* Close an ODP packet IO worker instance.
*
* @param id ODP packet IO worker handle.
*
* @return 0 on success, -1 on failure.
*/
int odp_pktio_worker_close(odp_pktio_worker_t id);
@endcode
@subsection odp_pktio_worker_recv odp_pktio_worker_recv
@code
/*
* Receive packets directly from the packet IO device.
*
* @param id ODP packet IO worker handle.
* @param[out] pkt_table[] Storage for received packets.
* @param len Length of pkt_table[], i.e. max number of pkts to
* receive.
*
* @return Number of packets received or -1 on error.
*/
int odp_pktio_worker_recv(odp_pktio_worker_t id, odp_packet_t pkt_table[],
unsigned len);
@endcode
Receive up to len packets directly from the pktio device and store them in
pkt_table. This call will return only as many packets as are immediately
available from the interface.
When accessing the same interface from multiple cores, packet order is not
maintained.
This routine may only be used on pktio interfaces that have no CoS rules
configured (and the default CoS is ODP_COS_INVALID).
@subsection odp_pktio_worker_send odp_pktio_worker_send
@code
/*
* Send packets directly via the pktio handle.
*
* @param id ODP packet IO worker handle.
* @param pkt_table[] Array of packets to send.
* @param len Length of pkt_table[].
*
* @return Number of packets sent or -1 on error.
*/
int odp_pktio_worker_send(odp_pktio_worker_t id, odp_packet_t pkt_table[],
unsigned len);
@endcode
Send packets directly via the pktio handle. The return value indicates the
number of packets that the implementation was able to immediately accept for
transmission. On function return those packets have been queued for
transmission but the actual transmission may not yet have occurred. During the
call ownership of the packets is transferred from the application to the
implementation, making it the responsibility of the implementation to free the
packets.
@subsection odp_packet_outq odp_packet_outq
@code
/*
* Get the output queue for a packet.
*
* @param pkt ODP packet buffer handle.
*
* @return output queue or ODP_QUEUE_INVALID on error.
*/
odp_queue_t odp_packet_outq(odp_packet_t pkt);
@endcode
This routine returns the output queue for a packet. This is a shorthand
equivalent of calling odp_packet_get_input() followed by odp_pktio_outq_getdef()
to determine the output queue based on the interface the packet was received on.
@section examples Usage Examples
This section contains some example code sequences demonstrating expected usage
of the APIs defined here. This is pseudo-ish code rather than something that
will actually compile, for real usable examples check the examples in the ODP
source repository.
Each example has two sections of code, an "Initialisation" section that is
expected to be run once from a single thread (e.g. the "main" thread) and a
"Worker" section that would be run from each worker thread.
@subsection exsched Single Interface via Scheduler
@subsubsection Initialisation
@code
odp_cos_t default_cos = odp_cos_create(..);
odp_buffer_pool_t pkt_pool = odp_buffer_pool_create(..);
odp_queue_t inq = odp_queue_create(..);
odp_cos_set_pool(default_cos, pkt_pool);
odp_cos_set_queue(default_cos, inq);
odp_pktio_t pktio = odp_pktio_open("eth0", default_cos);
/* additional pktios could be opened here, passing the same default_cos */
@endcode
@subsubsection Worker
@code
while (1) {
pkt = odp_schedule();
outq = odp_packet_outq(pkt);
buf = odp_buffer_from_packet(pkt);
/* do stuff.. */
odp_queue_enq(outq, buf);
}
@endcode
@subsection exdirect Direct Send/Receive
@subsubsection Initialisation
@code
odp_pktio_t pktio = odp_pktio_open("eth0", ODP_COS_INVALID);
@endcode
No buffer pool is associated with the pktio interface as it will only be
accessed by direct polling, in which case pools and buffers are managed
within the application.
@subsubsection Worker
@code
odp_pktio_worker_t io = odp_pktio_open_worker("eth0");
while (1) {
num_pkts = odp_pktio_worker_recv(io, pkt_tbl, ARRAY_SIZE(pkt_tbl));
odp_pktio_worker_send(io, pkt_tbl, num_pkts);
}
@endcode
@section implementation Implementation Considerations
Included here should be acceleration considerations for HW
implementation/offloads
associated with the defined functions, as well as suggested SW approaches for
efficient implementation of the defined function. Also included are global data
structures defined/referenced by the defined functions.
@section verification Verification
This section describes the verification/test cases needed to ensure that the
defined functionality is implemented correctly and performs adequately. This
should be at a level of detail such that the programmer tasked with writing
test scripts/programs to verify the implementation(s) of the defined functions
can be written to ensure that all relevant functional variants and
error/exception
cases are properly tested.
*/_______________________________________________
lng-odp mailing list
[email protected]
http://lists.linaro.org/mailman/listinfo/lng-odp
