Hi,
Yes, I believe your proposal of the double (minus 1) size array and the
documentation change is the best. I won’t write the function to do the
circulation, then.
How this could be abstracted:
odp_pktin_queue_t queues[] = {q0, q1, q2, q3};
odp_pktin_queue_rot_ctx_t rotctx;
odp_pktin_queue_rot_ctx_init(&rotctx, queues, sizeof(queues)/sizeof(*queues));
for (;;)
{
odp_pktin_queue_rot_ctx_rot(&rotctx);
ret = odp_pktin_recv_mq_tmo(odp_pktin_queue_rot_ctx_get(&rotctx),
odp_pktin_queue_rot_ctx_size(&rotctx), …);
/* process packets */
}
odp_pktin_queue_rot_ctx_free(&rotctx);
…and then rotctx would perform dynamic memory allocation with malloc() to
allocate an oversized array (dynamic memory allocation could be avoided if
there’s an upper limit for the number of queues).
But I don’t believe that abstraction for such a simple task is really
beneficial. Allocating the oversized array directly is IMO the best option.
From: Savolainen, Petri (Nokia - FI/Espoo)
Sent: Tuesday, March 08, 2016 11:17 AM
To: Tilli, Juha-Matti (Nokia - FI/Espoo); EXT Bill Fischofer
Cc: EXT Bala Manoharan; LNG ODP Mailman List
Subject: RE: [lng-odp] [API-NEXT PATCH 1/2] api: pktio: add recv from multiple
queues with tmo
Hi,
To keep it simple I propose (see under) to define only that application may
help in fairness by queue handle circulation. Application can circulate handles
efficiently (without any write) by holding handles in double size array and
advancing the start index. Additional function would need to move (write)
handles in the array. We can change it later if need more abstraction (==
additional function).
-Petri
queues[q0, q1, q2, q3, q0, q1, q2];
i = 0;
while (1) {
odp_pktin_recv_mq_tmo(&queues[i], 4, …)
if (i == 3)
i = 0;
else
i++;
}
/**
* Receive packets directly from multiple interface input queues
*
* Receives up to 'num' packets from one of the specified pktio interface input
* queues. The index of the source queue is stored into 'from' output
* parameter. If there are no packets available on any of the queues, waits for
* packets depeding on 'wait' parameter value. Returns the number of packets
* received.
*
* When an input queue has been configured with 'op_mode' value
* ODP_PKTIO_OP_MT_UNSAFE, the operation is optimized for single thread
* operation and the same queue must not be accessed simultaneously from
* multiple threads.
*
* It is implementation specific in which order the queues are checked for
* packets. Application may improve fairness of queue service levels by
* circulating queue handles between consecutive calls (e.g. [q0, q1, q2, q3] ->
* [q1, q2, q3, q0] -> [q2, q3, ...).
*
* @param queues[] Packet input queue handles for receiving packets
* @param num_q Number of input queues
* @param[out] from Pointer for output of the source queue index. Ignored
* when NULL.
* @param[out] packets[] Packet handle array for output of received packets
* @param num Maximum number of packets to receive
* @param wait Wait time specified as as follows:
* * ODP_PKTIN_NO_WAIT: Do not wait
* * ODP_PKTIN_WAIT: Wait infinitely
* * Other values specify the minimum time to wait.
* Use odp_pktin_wait_time() to convert nanoseconds
* to a valid parameter value. Wait time may be
* rounded up a small, platform specific amount.
*
* @return Number of packets received
* @retval <0 on failure
*/
int odp_pktin_recv_mq_tmo(const odp_pktin_queue_t queues[], unsigned num_q,
unsigned *from, odp_packet_t
packets[], int num,
uint64_t wait);
From: Tilli, Juha-Matti (Nokia - FI/Espoo)
Sent: Monday, March 07, 2016 11:23 AM
To: EXT Bill Fischofer
<[email protected]<mailto:[email protected]>>; Savolainen,
Petri (Nokia - FI/Espoo)
<[email protected]<mailto:[email protected]>>
Cc: EXT Bala Manoharan
<[email protected]<mailto:[email protected]>>; LNG ODP Mailman
List <[email protected]<mailto:[email protected]>>
Subject: RE: [lng-odp] [API-NEXT PATCH 1/2] api: pktio: add recv from multiple
queues with tmo
Hello,
I suggest adding another function:
void odp_pktin_mq_ensure_fairness(odp_pktin_queue_t queues[], unsigned num_q);
Then instead of this:
for (;;)
{
ret = odp_pktin_recv_mq_tmo(queues, sizeof(queues)/sizeof(*queues), NULL,
packets, sizeof(packets)/sizeof(*packets), ODP_PKTIN_WAIT);
/* process packets */
}
… the application would do this in a loop:
for (;;)
{
odp_pktin_mq_ensure_fairness(queues, sizeof(queues)/sizeof(*queues));
ret = odp_pktin_recv_mq_tmo(queues, sizeof(queues)/sizeof(*queues), NULL,
packets, sizeof(packets)/sizeof(*packets), ODP_PKTIN_WAIT);
/* process packets */
}
The actual “ensure fairness” implementation could be left unspecified, but for
the linux-generic, it would be implemented by simply rotating from {q1, q2, q3,
q4} to {q2, q3, q4, q1}. For other implementations, it might be even a no-op.
The idea is that “ensure fairness” would be called once before each recv_mq_tmo
call, and if this is done, each input queue would have approximately the same
priority when averaged over time. What ensure_fairness can do is to adjust the
order of the queues in the array, or adjust the queues themselves.
odp_pktin_recv_mq_tmo cannot do the ensure fairness step, because the queues
argument is a const argument.
Note that this proposal doesn’t require the recv_mq_tmo call to specify the
priority as 0, 1, 2, 3, … because the actual implementations of the recv_mq_tmo
and ensure_fairness calls would be left unspecified.
An alternative to this would be to change the queues argument to a non-const
argument so that recv_mq_tmo could do the ensure_fairness step, but I don’t
like this solution much. In my opinion, it is much better to have the queues
argument as a const argument.
From: EXT Bill Fischofer [mailto:[email protected]]
Sent: Sunday, March 06, 2016 11:13 AM
To: Savolainen, Petri (Nokia - FI/Espoo)
Cc: EXT Bala Manoharan; Tilli, Juha-Matti (Nokia - FI/Espoo); LNG ODP Mailman
List
Subject: Re: [lng-odp] [API-NEXT PATCH 1/2] api: pktio: add recv from multiple
queues with tmo
We will discuss this during tomorrow's deep-dive covering remaining Monarch RC2
items. One further comment regarding the proposed implementation of this new
API is that it should be using the ODP time APIs rather than nanosleep.
On Fri, Mar 4, 2016 at 9:09 AM, Savolainen, Petri (Nokia - FI/Espoo)
<[email protected]<mailto:[email protected]>> wrote:
From: EXT Bala Manoharan
[mailto:[email protected]<mailto:[email protected]>]
Sent: Friday, March 04, 2016 2:39 PM
To: Tilli, Juha-Matti (Nokia - FI/Espoo)
<[email protected]<mailto:[email protected]>>
Cc: Savolainen, Petri (Nokia - FI/Espoo)
<[email protected]<mailto:[email protected]>>; LNG ODP
Mailman List <[email protected]<mailto:[email protected]>>
Subject: Re: [lng-odp] [API-NEXT PATCH 1/2] api: pktio: add recv from multiple
queues with tmo
On 4 March 2016 at 17:47, Tilli, Juha-Matti (Nokia - FI/Espoo)
<[email protected]<mailto:[email protected]>> wrote:
Hi,
The current algorithm indeed is unfair, because queue 0 has the highest
priority. There are at least two solutions to this:
1. Leave the priority order unspecified
2. Specify that queues are checked in the order they appear in the array
I kind of like solution (2), because it allows the application to behave in a
round-robin manner by altering the order of the queues between the calls:
- odp_pktin_recv_mq_tmo({q1, q2, q3, q4}, …)
- odp_pktin_recv_mq_tmo({q2, q3, q4, q1}, …)
- odp_pktin_recv_mq_tmo({q3, q4, q1, q2}, …)
- odp_pktin_recv_mq_tmo({q4, q1, q2, q3}, …)
…
Then essentially the priority of all queues averages out to have approximately
the same average priority.
So, you should perhaps consider in the API spec that the priority order could
be defined to be 0, 1, 2, …. I cannot foresee any implementation which would
use a different priority order.
Agreed. We can improve the documentation to specify that priority is in the
order of input queue in the array with '0' being highest priority.
Regards,
Bala
I'd add in a next step (if needed and feasible).
The algorithm above works for fairness regardless if the first or any other
queue has highest priority. I'd like to first get verification that strict
priority specification would not cause performance issues. E.g. on some
platform it may optimize performance, if the next recv_mq() call continues from
the same queue the last finished, or any other relation between queues (e.g. if
two queues point to related interfaces, handle those together: first all netmap
interfaces, then all sockets, ...).
-Petri
_______________________________________________
lng-odp mailing list
[email protected]<mailto:[email protected]>
https://lists.linaro.org/mailman/listinfo/lng-odp
_______________________________________________
lng-odp mailing list
[email protected]
https://lists.linaro.org/mailman/listinfo/lng-odp
