On Wed, Oct 14, 2015 at 02:32:37PM +0300, Nikita Kalyazin wrote:
> This series is a proposal of multiqueue API for ODP.
> Modern network adapters (both physical and virtual) support multiple
> hardware queues. Assigning separate CPU core for processing of each
> queue allows to scale throughput (in best case - linearly).
>
Thanks for the contribution, we do need something like this, in
particular hash based distribution has been on the todo list for a
little while. I've not reviewed the implementation in detail but have
focused on the API changes specifically and have a few initial thoughts
below.
One thing that is missing here is a method for the application to
configure how packets are distributed, i.e. which fields in the packet
are used when calculating the flow hash (not necessarily the hash
algorithm itself, that can be implementation defined).
For example with the netmap implementation here, if you have a
compatible interface, I assume the user can control distribution to RX
queues using something like;
ethtool --config-ntuple rx-flow-hash esp4
But it would be better if this were configurable via the ODP API. Petri
had a suggestion a while back related to this to add a structure to the
odp_pktio_params_t;
enum odp_pktio_input_hash {
/** No specific fields defined */
ODP_PKTIN_HASH_NONE = 0,
/** IPv4/v6 addresses */
ODP_PKTIN_HASH_IP,
/** UDP ports and IPv4/v6 addresses */
ODP_PKTIN_HASH_UDP_IP,
/** TCP ports and IPv4/v6 addresses */
ODP_PKTIN_HASH_TCP_IP
};
As for RX queue creation, rather than proposed sequence that looks like
this:
odp_pktio_max_num_queues()
odp_pktio_configure()
foreach inq
odp_queue_create()
How about:
int odp_pktio_inq_create(odp_pktio_t pktio, const char *name,
const odp_queue_param_t *param,
odp_queue_t queues[], int num);
Which would return the number of queues created, and ensure that all
queues had the same parameters. Or, maybe it's time to start using
odp_queue_group_t, which we have always intended to be a group of queues
that all have the same properties.
I'm not keen on the odp_pktio_recv/send_queue() APIs as the queues
referred to there are not ODP queues so it makes things a bit less clear,
and in general we should be moving away from using the direct _send/_recv
APIs. If we're explicitly using queues then we should use the queue and/or
scheduler APIs.
Are you able to join the meeting on Tuesday to discuss?
http://www.opendataplane.org/meetings/
> This series should be considered a prototype.
> This implementation of multiqueue API is not complete (not all the
> code has been migrated to multiqueue API). Existing packet IO API has
> been left untouched and reimplemented using mutliqueue API (queue 0 is
> always used in such cases). The existing API is supposed to be
> removed in the future, as soon as all the remaining code migrates to
> the multiqueue API.
> Since single lock for all the queues makes all pktio Rx and Tx calls
> sequential, per queue locks have been added.
> For the purpose of evaluation, multiqueue support has been added to
> netmap pktio.
>
> Patch 1 adds multiqueue prototypes of pktio API functions.
> Patches 2-4 contain common implementation of the multiqueue API.
> Patch 5 adds multiqueue support in netmap pktio.
> Patch 6 moves odp_generator to multiqueue API.
>
> In our tests (odp_generator sends packets, odp_generator receives
> packets), the following results have been obtained:
> +----------+------------------------+-------+
> | # queues | kpps (64-byte packets) | boost |
> +----------+------------------------+-------+
> | 1 | 853 | 1.00x |
> | 2 | 1404 | 1.64x |
> | 4 | 2374 | 2.78x |
> | 8 | 4345 | 5.09x |
> +----------+------------------------+-------+
>
> Nikita Kalyazin (6):
> api: pktio: add multiqueue API
> linux-generic: pktio: add per queue pktio locks
> linux-generic: pktio: implement multiqueue API (part I)
> linux-generic: pktio: imlement multiqueue API (part II)
> linux-generic: pktio: multiqueue support in netmap
> example: generator: use multiqueue API
>
> example/generator/odp_generator.c | 63 +++-
> include/odp/api/config.h | 7 +-
> include/odp/api/packet_io.h | 118 ++++++
> .../linux-generic/include/odp_packet_io_internal.h | 19 +-
> platform/linux-generic/include/odp_packet_netmap.h | 6 +-
> .../linux-generic/include/odp_queue_internal.h | 2 +
> .../linux-generic/include/odp_schedule_internal.h | 1 +
> platform/linux-generic/odp_packet_io.c | 405
> +++++++++++++++++----
> platform/linux-generic/odp_schedule.c | 26 +-
> platform/linux-generic/pktio/netmap.c | 218 +++++++++--
> 10 files changed, 725 insertions(+), 140 deletions(-)
>
> --
> 2.5.3
>
_______________________________________________
lng-odp mailing list
[email protected]
https://lists.linaro.org/mailman/listinfo/lng-odp
