On Wed, Feb 14, 2018 at 5:28 AM, Sowmini Varadhan
> This is version 2 of the series at
> Review comments addressed
> Patch 4:
> - make sure to always sock_put m_rs even if there is no znotifier.
> - major rewrite of notification, resulting in much simplification.
> Patch 5:
> - remove unused data_len argument to rds_rm_size;
> - unmap as necessary if we fail in the middle of zerocopy setup
> Patch 7:
> - restructured do_recv_completion to avoid excessive code re-indent
> - on-stack allocation of cmsghdr for cookie in do_sendmsg
> - Additional verification: Verify ncookies <= MAX_.., verify ret ==
> ncookies * sizeof(uint32_t)
> A brief overview of this feature follows.
> This patch series provides support for MSG_ZERCOCOPY
> on a PF_RDS socket based on the APIs and infrastructure added
> by f214f915e7db ("tcp: enable MSG_ZEROCOPY")
> For single threaded rds-stress testing using rds-tcp with the
> ixgbe driver using 1M message sizes (-a 1M -q 1M) preliminary
> results show that there is a significant reduction in latency: about
> 90 usec with zerocopy, compared with 200 usec without zerocopy.
> This patchset modifies the above for zerocopy in the following manner.
> - if the MSG_ZEROCOPY flag is specified with rds_sendmsg(), and,
> - if the SO_ZEROCOPY socket option has been set on the PF_RDS socket,
> application pages sent down with rds_sendmsg are pinned. The pinning
> uses the accounting infrastructure added by a91dbff551a6 ("sock: ulimit
> on MSG_ZEROCOPY pages"). The message is unpinned when all references
> to the message go down to 0, and the message is freed by rds_message_purge.
> A multithreaded application using this infrastructure must send down
> a unique 32 bit cookie as ancillary data with each sendmsg invocation.
> The format of this ancillary data is described in Patch 5 of the series.
> The cookie is passed up to the application on the sk_error_queue when
> the message is unpinned, indicating to the application that it is now
> safe to free/reuse the message buffer. The details of the completion
> notifiction are provided in Patch 4 of this series.
> Sowmini Varadhan (7):
> skbuff: export mm_[un]account_pinned_pages for other modules
> rds: hold a sock ref from rds_message to the rds_sock
> sock: permit SO_ZEROCOPY on PF_RDS socket
> rds: support for zcopy completion notification
> rds: zerocopy Tx support.
> selftests/net: add support for PF_RDS sockets
> selftests/net: add zerocopy support for PF_RDS test case
> include/linux/skbuff.h | 3 +
> include/uapi/linux/errqueue.h | 2 +
> include/uapi/linux/rds.h | 1 +
> net/core/skbuff.c | 6 +-
> net/core/sock.c | 25 +++---
> net/rds/af_rds.c | 2 +
> net/rds/message.c | 132 ++++++++++++++++++++++++++-
> net/rds/rds.h | 17 ++++-
> net/rds/recv.c | 2 +
> net/rds/send.c | 51 ++++++++---
> tools/testing/selftests/net/msg_zerocopy.c | 133 ++++++++++++++++++++++++++-
> 11 files changed, 339 insertions(+), 35 deletions(-)
A few inline suggestions in patch 5/7 were already discussed in that thread.
These will be cleaned up in a follow-on patch.
The fragile use of skb->cb in patch 4/7 should also be addressed there.
With that caveat, for the series:
Acked-by: Willem de Bruijn <will...@google.com>