I think it should be OK to use ordered queues instead of atomic queues, e.g. for sequence number allocation (which will need an atomic variable to hold the seqno counter). Packet ingress order will be maintained but might not always correspond to sequence number order. This is not a problem a the sliding window in the replay protection will take care of that, the sliding window could be hundreds of entries (sequence numbers) large (each will only take one bit). Packet ingress order is the important characteristic that must be maintained. The IPsec sequence number is not used for packet ordering and order restoration, it is only used for replay protection.
Someone with a platform which supports both ordered and atomic scheduling could benchmark both designs and see how performance scales when using ordered queues (and that atomic fetch_and_add) for some relevant traffic patterns. On 8 May 2015 at 13:53, Bill Fischofer <[email protected]> wrote: > Jerrin, > > Can you propose such a set of APIs for further discussion? This would be > good to discuss at the Tuesday call. > > Thanks. > > Bill > > On Fri, May 8, 2015 at 12:07 AM, Jacob, Jerin < > [email protected]> wrote: > >> >> I agree with Ola here on preserving the ingress order. >> However, I have experienced same performance issue as Nikhil pointed out >> (atomic queues have too much overhead for short critical section) >> >> I am not sure about any other HW but Cavium has support for >> introducing the critical section while maintain the ingress order as a HW >> scheduler feature. >> >> IMO, if such support is available in other HW then >> odp_schedule_ordered_lock()/unlock() >> kind of API will solve the performance issue for the need for short >> critical section in ordered flow. >> >> /Jerin. >> >> >> From: lng-odp <[email protected]> on behalf of Ola >> Liljedahl <[email protected]> >> Sent: Thursday, May 7, 2015 9:06 PM >> To: [email protected] >> Cc: [email protected] >> Subject: Re: [lng-odp] Query regarding sequence number update in IPSEC >> application >> >> >> Using atomic queues will preserve the ingress order when allocating and >> assigning the sequence number. Also you don't need to use an expensive >> atomic operation for updating the sequence number as the atomic queue and >> scheduling will provide mutual exclusion. >> >> >> If the packets that require a sequence number came from parallel or >> ordered queues, there would be no guarantee that the sequence numbers would >> be allocated in packet (ingress) order. Just using an atomic operation >> (e.g. fetch_and_add or similar) only guarantees proper update of the >> sequence number variable, not any specific ordering. >> >> >> If you are ready to trade absolute "correctness" for performance, you >> could use ordered or may even parallel (questionable for other reasons) >> queues and then allocate the sequence number using an atomic fetch_and_add. >> Sometimes packets egress order will then not match the sequence number >> order (for a flow/SA). For IPSec, this might affect the replay window check >> & update at the receiving end but as the replay protection uses a sliding >> window of sequence numbers (to handle misordered packets), there might not >> be any adverse effects in practice. The most important aspect is probably >> to preserve original packet order. >> >> >> -- Ola >> >> >> On 6 May 2015 at 11:29, [email protected] < >> [email protected]> wrote: >> >> >> Hi, >> >> In IPSEC example application, queues are used to update the sequence >> number. I was wondering why we have used queues to update sequence number >> which will add to scheduling delays and adversely hit the performance >> throughput. Is there any specific advantage of using queues over atomic >> variables. >> >> Thanks in advance >> Nikhil >> >> >> _______________________________________________ >> lng-odp mailing list >> [email protected] >> https://lists.linaro.org/mailman/listinfo/lng-odp >> >> >> >> _______________________________________________ >> lng-odp mailing list >> [email protected] >> https://lists.linaro.org/mailman/listinfo/lng-odp >> > >
_______________________________________________ lng-odp mailing list [email protected] https://lists.linaro.org/mailman/listinfo/lng-odp
