On Sat, Jun 16, 2018 at 7:21 PM, Vishal Deep Ajmera < [email protected]> wrote: > > OVS reads packets in batches from a given port and packets in the > batch are subjected to potentially 3 levels of lookups to identify > the datapath megaflow entry (or flow) associated with the packet. > Each megaflow entry has a dedicated buffer in which packets that match > the flow classification criteria are collected. This buffer helps OVS > perform batch processing for all packets associated with a given flow. > > Each packet in the received batch is first subjected to lookup in the > Exact Match Cache (EMC). Each EMC entry will point to a flow. If the > EMC lookup is successful, the packet is moved from the rx batch to the > per-flow buffer. > > Packets that did not match any EMC entry are rearranged in the rx batch > at the beginning and are now subjected to a lookup in the megaflow cache. > Packets that match a megaflow cache entry are *appended* to the per-flow > buffer. > > Packets that do not match any megaflow entry are subjected to slow-path > processing through the upcall mechanism. This cannot change the order of > packets as by definition upcall processing is only done for packets > without matching megaflow entry. > > The EMC entry match fields encompass all potentially significant header > fields, typically more than specified in the associated flow's match > criteria. Hence, multiple EMC entries can point to the same flow. Given > that per-flow batching happens at each lookup stage, packets belonging > to the same megaflow can get re-ordered because some packets match EMC > entries while others do not. > > The following example can illustrate the issue better. Consider > following batch of packets (labelled P1 to P8) associated with a single > TCP connection and associated with a single flow. Let us assume that > packets with just the ACK bit set in TCP flags have been received in a > prior batch also and a corresponding EMC entry exists. > > 1. P1 (TCP Flag: ACK) > 2. P2 (TCP Flag: ACK) > 3. P3 (TCP Flag: ACK) > 4. P4 (TCP Flag: ACK, PSH) > 5. P5 (TCP Flag: ACK) > 6. P6 (TCP Flag: ACK) > 7. P7 (TCP Flag: ACK) > 8. P8 (TCP Flag: ACK) > > The megaflow classification criteria does not include TCP flags while > the EMC match criteria does. Thus, all packets other than P4 match > the existing EMC entry and are moved to the per-flow packet batch. > Subsequently, packet P4 is moved to the same per-flow packet batch as > a result of the megaflow lookup. Though the packets have all been > correctly classified as being associated with the same flow, the > packet order has not been preserved because of the per-flow batching > performed during the EMC lookup stage. This packet re-ordering has > performance implications for TCP applications. > > This patch preserves the packet ordering by performing the per-flow > batching after both the EMC and megaflow lookups are complete. As an > optimization, packets are flow-batched in emc processing till any > packet in the batch has an EMC miss. > > A new flow map is maintained to keep the original order of packet > along with flow information. Post fastpath processing, packets from > flow map are *appended* to per-flow buffer. > > Signed-off-by: Vishal Deep Ajmera <[email protected]> > Co-authored-by: Venkatesan Pradeep <[email protected]> > Signed-off-by: Venkatesan Pradeep <[email protected]> > --- > lib/dpif-netdev.c | 79 ++++++++++++++++++++++++++++++++++++++++++++++--------- > 1 file changed, 66 insertions(+), 13 deletions(-)
Thanks for the patch! I didn't review the patch, but just have a question: does this issue exist only for dpif-netdev, or does kernel datapath has same issue? _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
