Hi, In addition configuring the hashing on the pktio interface as Petri suggested, you can also configure hashing on CoS. By configuring hashing on CoS, you can effectively configure hashing for a particular flow.
Regards, Bala On 4 January 2018 at 20:51, Oriol Arcas <[email protected]> wrote: > Bill, Petri, Bogdan, > > Thank you for your fast feedback. It's been incredibly instructive. We were > looking for something like the input hash that Petri points out, even at > the price of not having classification (which we could implement manually). > > I'll dive into the details of queue scheduling and sticking to a CPU. > > Thank you all. > > -- > Oriol Arcas > Software Engineer > Starflow Networks > > On Thu, Jan 4, 2018 at 11:01 AM, Bogdan Pricope <[email protected] > > > wrote: > > > I guess, the issue is not how to hash traffic in different scheduled > > queues but how to lock a scheduled queue to a single thread (core): > > sched.sync guarantees that at one moment one queue is scheduled to a > > single thread but not on the same thread all the times - this may be > > enough for some implementations (to avoid some locks) but not enough > > for others. > > > > One problem is that all pktio sched queues are assigned to a single > > sched group that is assigned to a single group of threads/cores. If I > > understand correctly, Bill suggests classification + sched queues, > > where each queue is assigned to a different sched group that is > > assigned to a single thread/core. > > > > Other idea is to use direct mode (+ RSS), where each worker is polling > > from the its own pktin all the time (odp_pktin_recv()). > > > > On 4 January 2018 at 10:24, Savolainen, Petri (Nokia - FI/Espoo) > > <[email protected]> wrote: > > > > > > > > >> -----Original Message----- > > >> From: lng-odp [mailto:[email protected]] On Behalf Of > > Oriol > > >> Arcas > > >> Sent: Wednesday, January 03, 2018 7:12 PM > > >> To: LNG ODP Mailman List <[email protected]> > > >> Subject: [lng-odp] RSS in ODP > > >> > > >> Hello and happy new year, > > >> > > >> In our company we are looking into scaling the odp_schedule() calls. > > >> Currently we are manually doing Receive Side Scaling, which requires > one > > >> CPU to receive all the packets and pass them to other worker CPU in a > > >> flow-deterministic way (i.e., not spreading the packets from a TCP > flow > > to > > >> different CPUs). Obviously this is a bottleneck. > > >> > > >> It would be great if ODP had optional RSS policies, which ultimately > > would > > >> assign packets from the same flow to a single thread in the schedule > > group > > >> (usually hashing the address tuple). Would this probably mean having > > >> dedicated queues? > > >> > > >> I don't know if there is something similar in ODP already which I have > > >> missed. I'll thank any feedback! > > >> > > >> Best regards, > > >> > > >> -- > > >> Oriol Arcas > > >> Software Engineer > > >> Starflow Networks > > > > > > > > > Our l2fwd test application (odp_l2fwd.c) configures packet input > > hashing, which is in practice RSS, but could be also some other > > implementation defined packet input hash function. You can take a look > from > > there. The same hash configuration is possible for both direct pktin > queues > > and scheduled event queues. For scheduled queues you would enable it > > something like this: > > > > > > /* Normal interface open and config steps */ > > > pktio = odp_pktio_open(dev, pool, &pktio_param); > > > odp_pktio_config(pktio, &config); > > > > > > /* > > > * Setup packet input hashing into scheduled event queues > > > */ > > > if (num_rx_queues > capa.max_input_queues) > > > num_rx_queues = capa.max_input_queues; > > > > > > odp_pktin_queue_param_init(&pktin_param); > > > > > > pktin_param.queue_param.sched.prio = ODP_SCHED_PRIO_DEFAULT; > > > pktin_param.queue_param.sched.sync = ODP_SCHED_SYNC_ATOMIC; > > > pktin_param.queue_param.sched.group = ODP_SCHED_GROUP_ALL; > > > pktin_param.hash_enable = 1; > > > pktin_param.hash_proto.proto.ipv4_udp = 1; > > > pktin_param.num_queues = num_rx_queues; > > > > > > if (odp_pktin_queue_config(pktio, &pktin_param)) > > > return -1; > > > > > > /* Optionally, see which event queues has been created by the previous > > call. > > > * May e.g. want to set queue contexts here. > > > */ > > > if (odp_pktin_event_queue(pktio, rx_queues, num_rx_queues) != > > num_rx_queues) > > > return -1; > > > > > > /* Starts packet input */ > > > odp_pktio_start(pktio); > > > > > > /* Use scheduler to receive packets ...*/ > > > > > > > > >
