On Sun, Sep 4, 2016 at 2:50 AM, Michał Purzyński <michalpurzyns...@gmail.com> wrote: > My IXGBE data path journey made me ask questions, like the one above. > Basically (I'm skipping a few steps here): > > 1. packet arrives to the card and is verified by MAC > 2. packet is placed in the card's FIFO (which is small) > 3. lots of steps here but finally card does DMA into one of receive buffers > 4. are these buffers shared with skb->data ? Or does 82599 copy bits > between rx buffers and skb->data ? > > Given that dma_map_single(rx_ring->dev, skb->data,...) is called in > ixgbe_alloc_maped_skb() I would say "card does DMA into SKB" (excuse my > oversimplification) but I'd like to verify it, especially that > ixgbe_alloc_mapped_page() can be used instead, depending on > CONFIG_IXGBE_DISABLE_PACKET_SPLIT
So what you are describing sounds like an older ixgbe driver. We used to write either all the data into skb->data or the packet header. If the Rx path is using ixgbe_alloc_mapped_skb() then yes, we are writing directly to skb->data. The cost for doing that though is that we have to leave the skb allocated prior to actually receiving the Rx buffer which can be pretty expensive since the skb has to reserve the space for the sk_buff plus the skb_shared info. Several years ago I rewrote that so that we received the entire frame into one or more page based buffers. After that we copy the header of the frame from the page and into skb->data. The advantage to that approach is that we don't have to map/unmap the Rx buffers from the device which can be very expensive when an IOMMU is enabled. > I'm trying to understand where data copying occurs, because it might be > impossible to go with some clever memory mappings. It depends on the driver being used. If you are using an older one with ixgbe_alloc_mapped_skb then the data copy will occur when you transition to user space. So the copy occurs up at the socket level. If you are using a newer driver there is a small copy in the driver, and then the data gets copied up at the socket level. > A side question - why is the FIFO buffer that small? What's stopping us > from having it (theoretically) say 1GB? Some (super expensive) capture > cards do this, but they have other challenges. I think you just answered your own question. Adding memory would add to the overall cost of the part making it more expensive. > Of course, X520 is not a capture card (but it's very efficient doing it, > anyway). I do not believe in a dedicated hardware anymore. If you are looking at just doing straight capture and analysis another option you might look at would be something like PF_RING as I believe it is meant to address some of what you are pointing out by avoiding the copy to user-space. - Alex ------------------------------------------------------------------------------ _______________________________________________ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
