On 19 October 2015 at 19:16, Nikita Kalyazin <[email protected]> wrote:
> Hi Stuart,
>
>
> Thanks for your feedback.
>
> > One thing that is missing here is a method for the application to
> > configure how packets are distributed, i.e. which fields in the packet
> > are used when calculating the flow hash (not necessarily the hash
> > algorithm itself, that can be implementation defined).
> >
> > For example with the netmap implementation here, if you have a
> > compatible interface, I assume the user can control distribution to RX
> > queues using something like;
> >
> > ethtool --config-ntuple rx-flow-hash esp4
> >
> > But it would be better if this were configurable via the ODP API. Petri
> > had a suggestion a while back related to this to add a structure to the
> > odp_pktio_params_t;
> >
> > enum odp_pktio_input_hash {
> > /** No specific fields defined */
> > ODP_PKTIN_HASH_NONE = 0,
> > /** IPv4/v6 addresses */
> > ODP_PKTIN_HASH_IP,
> > /** UDP ports and IPv4/v6 addresses */
> > ODP_PKTIN_HASH_UDP_IP,
> > /** TCP ports and IPv4/v6 addresses */
> > ODP_PKTIN_HASH_TCP_IP
> > };
> Thanks, I plan to think about it.
> Btw, NICs provide wide variety of settings regarding distribution of
> packets across the queues. Is it supposed to extend the proposed
> structure to support more flexible configuration?
>
> > How about:
> >
> > int odp_pktio_inq_create(odp_pktio_t pktio, const char *name,
> > const odp_queue_param_t *param,
> > odp_queue_t queues[], int num);
> >
> > Which would return the number of queues created, and ensure that all
> > queues had the same parameters. Or, maybe it's time to start using
> > odp_queue_group_t, which we have always intended to be a group of queues
> > that all have the same properties.
> Good point. I'll consider the cleaner approach.
>
> > I'm not keen on the odp_pktio_recv/send_queue() APIs as the queues
> > referred to there are not ODP queues so it makes things a bit less clear,
> > and in general we should be moving away from using the direct _send/_recv
> > APIs. If we're explicitly using queues then we should use the queue
> and/or
> > scheduler APIs.
> Right, these API names look a bit confusing, because same word is used to
> refer to both hardware NIC queues and ODP (software) queues.
>
[Alex] ODP queues are neither software nor hardware by definition, each
implementation is free to implement them as they see fit. Also PacketI/O
abstraction is not an abstraction for a NIC device. The ODP way to use
multiple queues for ingress is through the classifier. There was a former
proposal of enabling RSS at pktio level but it was not accepted. I also
noticed that this patch tries to introduce multiple default queues which is
kind of confusing - default traffic is all traffic which does not satisfy
classifier criteria. For output, multiple queues are used by the means of
TM api.
> However, I think it would be useful to preserve explicit send/recv APIs,
> since in many cases they might appear more efficient (odp-ovs is an example
> of such applications), and rename those to make them more obvious.
>
> > Are you able to join the meeting on Tuesday to discuss?
> >
> > http://www.opendataplane.org/meetings/
> Thanks, I'm going to join the meeting.
>
> --
>
> Best regards,
>
> Nikita Kalyazin,
> [email protected]
>
> Software Engineer
> Virtualization Group
> Samsung R&D Institute Russia
> Tel: +7 (495) 797-25-00 #3816
> Tel: +7 (495) 797-25-03
> Office #1501, 12-1, Dvintsev str.,
> Moscow, 127018, Russia
>
> On Fri, Oct 16, 2015 at 07:15:43PM +0100, Stuart Haslam wrote:
> > On Wed, Oct 14, 2015 at 02:32:37PM +0300, Nikita Kalyazin wrote:
> > > This series is a proposal of multiqueue API for ODP.
> > > Modern network adapters (both physical and virtual) support multiple
> > > hardware queues. Assigning separate CPU core for processing of each
> > > queue allows to scale throughput (in best case - linearly).
> > >
> >
> > Thanks for the contribution, we do need something like this, in
> > particular hash based distribution has been on the todo list for a
> > little while. I've not reviewed the implementation in detail but have
> > focused on the API changes specifically and have a few initial thoughts
> > below.
> >
> > One thing that is missing here is a method for the application to
> > configure how packets are distributed, i.e. which fields in the packet
> > are used when calculating the flow hash (not necessarily the hash
> > algorithm itself, that can be implementation defined).
> >
> > For example with the netmap implementation here, if you have a
> > compatible interface, I assume the user can control distribution to RX
> > queues using something like;
> >
> > ethtool --config-ntuple rx-flow-hash esp4
> >
> > But it would be better if this were configurable via the ODP API. Petri
> > had a suggestion a while back related to this to add a structure to the
> > odp_pktio_params_t;
> >
> > enum odp_pktio_input_hash {
> > /** No specific fields defined */
> > ODP_PKTIN_HASH_NONE = 0,
> > /** IPv4/v6 addresses */
> > ODP_PKTIN_HASH_IP,
> > /** UDP ports and IPv4/v6 addresses */
> > ODP_PKTIN_HASH_UDP_IP,
> > /** TCP ports and IPv4/v6 addresses */
> > ODP_PKTIN_HASH_TCP_IP
> > };
> >
> > As for RX queue creation, rather than proposed sequence that looks like
> > this:
> >
> > odp_pktio_max_num_queues()
> > odp_pktio_configure()
> > foreach inq
> > odp_queue_create()
> >
> > How about:
> >
> > int odp_pktio_inq_create(odp_pktio_t pktio, const char *name,
> > const odp_queue_param_t *param,
> > odp_queue_t queues[], int num);
> >
> > Which would return the number of queues created, and ensure that all
> > queues had the same parameters. Or, maybe it's time to start using
> > odp_queue_group_t, which we have always intended to be a group of queues
> > that all have the same properties.
> >
> > I'm not keen on the odp_pktio_recv/send_queue() APIs as the queues
> > referred to there are not ODP queues so it makes things a bit less clear,
> > and in general we should be moving away from using the direct _send/_recv
> > APIs. If we're explicitly using queues then we should use the queue
> and/or
> > scheduler APIs.
> >
> > Are you able to join the meeting on Tuesday to discuss?
> >
> > http://www.opendataplane.org/meetings/
> >
> > > This series should be considered a prototype.
> > > This implementation of multiqueue API is not complete (not all the
> > > code has been migrated to multiqueue API). Existing packet IO API has
> > > been left untouched and reimplemented using mutliqueue API (queue 0 is
> > > always used in such cases). The existing API is supposed to be
> > > removed in the future, as soon as all the remaining code migrates to
> > > the multiqueue API.
> > > Since single lock for all the queues makes all pktio Rx and Tx calls
> > > sequential, per queue locks have been added.
> > > For the purpose of evaluation, multiqueue support has been added to
> > > netmap pktio.
> > >
> > > Patch 1 adds multiqueue prototypes of pktio API functions.
> > > Patches 2-4 contain common implementation of the multiqueue API.
> > > Patch 5 adds multiqueue support in netmap pktio.
> > > Patch 6 moves odp_generator to multiqueue API.
> > >
> > > In our tests (odp_generator sends packets, odp_generator receives
> > > packets), the following results have been obtained:
> > > +----------+------------------------+-------+
> > > | # queues | kpps (64-byte packets) | boost |
> > > +----------+------------------------+-------+
> > > | 1 | 853 | 1.00x |
> > > | 2 | 1404 | 1.64x |
> > > | 4 | 2374 | 2.78x |
> > > | 8 | 4345 | 5.09x |
> > > +----------+------------------------+-------+
> > >
> > > Nikita Kalyazin (6):
> > > api: pktio: add multiqueue API
> > > linux-generic: pktio: add per queue pktio locks
> > > linux-generic: pktio: implement multiqueue API (part I)
> > > linux-generic: pktio: imlement multiqueue API (part II)
> > > linux-generic: pktio: multiqueue support in netmap
> > > example: generator: use multiqueue API
> > >
> > > example/generator/odp_generator.c | 63 +++-
> > > include/odp/api/config.h | 7 +-
> > > include/odp/api/packet_io.h | 118 ++++++
> > > .../linux-generic/include/odp_packet_io_internal.h | 19 +-
> > > platform/linux-generic/include/odp_packet_netmap.h | 6 +-
> > > .../linux-generic/include/odp_queue_internal.h | 2 +
> > > .../linux-generic/include/odp_schedule_internal.h | 1 +
> > > platform/linux-generic/odp_packet_io.c | 405
> +++++++++++++++++----
> > > platform/linux-generic/odp_schedule.c | 26 +-
> > > platform/linux-generic/pktio/netmap.c | 218 +++++++++--
> > > 10 files changed, 725 insertions(+), 140 deletions(-)
> > >
> > > --
> > > 2.5.3
> > >
> _______________________________________________
> lng-odp mailing list
> [email protected]
> https://lists.linaro.org/mailman/listinfo/lng-odp
>
_______________________________________________
lng-odp mailing list
[email protected]
https://lists.linaro.org/mailman/listinfo/lng-odp
