> The current IPoIB-UD implementation is limited IPoIB payload size to > 2048 through hard coding IPOIB_PACKET_SIZE. The implementation is > designed for kernel PAGE_SIZE equals or greater than 4K. If the kernel > PAGE_SIZE is equals to 2K, memory buffer allocation will be failure when > lack of large buffer of memory. However most of the Distros does support > PAGE_SIZE >= 4K. So this implementation has no problem for 2048 payload. > This implementation is simple but it prevents HCA device who does > support 4096 payload from performing, like IBM eHCA2.
Not sure I understand this. Is there any possible configuration of any architecture where Linux runs where PAGE_SIZE < 4096? > This patch allows IPoIB-UD MTU up to 4092 (4K - IPOIB_ENCAP_LEN) when > HCA can support 4K MTU. In this patch, APIs for S/G buffer allocation in > IPoIB-CM mode has been made generic so IPoIB-UD and IPoIB-CM can share > the S/G code. This approach seems overly complex to me, since it ends up going through all the CM buffer fragment bookkeeping for the simple UD path. However, I now realize that my earlier idea of allocating a scratch buffer for the GRH and just allocating a 4096 byte skb doesn't work, because the skb_shinfo ends up being allocated along with the buffer, so trying to allocate a 4096-byte skb will bloat the data past a single page, which is what we're trying to avoid. So how about the following? When using a UD MTU of 4096 with a page size of 4096, allocate an skb of size 44 for the GRH and ethertype, and then allocate a single page for the fragment list. This means that the IP packet will start nicely 16-byte aligned for free, and all the bookkeeping is very simple. - R. _______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
