+ Dave Miller, Jakub Kicinski, Paolo Abeni, Shuah Khan, [email protected]
On Mon, Jul 08, 2024 at 09:04:05PM +0000, [email protected] wrote: > From: Zijian Zhang <[email protected]> > > We update selftests/net/msg_zerocopy.c to accommodate the new mechanism, > cfg_notification_limit has the same semantics for both methods. Test > results are as follows, we update skb_orphan_frags_rx to the same as > skb_orphan_frags to support zerocopy in the localhost test. > > cfg_notification_limit = 1, both method get notifications after 1 calling > of sendmsg. In this case, the new method has around 17% cpu savings in TCP > and 23% cpu savings in UDP. > +---------------------+---------+---------+---------+---------+ > | Test Type / Protocol| TCP v4 | TCP v6 | UDP v4 | UDP v6 | > +---------------------+---------+---------+---------+---------+ > | ZCopy (MB) | 7523 | 7706 | 7489 | 7304 | > +---------------------+---------+---------+---------+---------+ > | New ZCopy (MB) | 8834 | 8993 | 9053 | 9228 | > +---------------------+---------+---------+---------+---------+ > | New ZCopy / ZCopy | 117.42% | 116.70% | 120.88% | 126.34% | > +---------------------+---------+---------+---------+---------+ > > cfg_notification_limit = 32, both get notifications after 32 calling of > sendmsg, which means more chances to coalesce notifications, and less > overhead of poll + recvmsg for the original method. In this case, the new > method has around 7% cpu savings in TCP and slightly better cpu usage in > UDP. In the context of selftest, notifications of TCP are more likely to > out of order than UDP, it's easier to coalesce more notifications in UDP. > The original method can get one notification with range of 32 in a recvmsg > most of the time. In TCP, most notifications' range is around 2, so the > original method needs around 16 recvmsgs to get notified in one round. > That's the reason for the "New ZCopy / ZCopy" diff in TCP and UDP here. > +---------------------+---------+---------+---------+---------+ > | Test Type / Protocol| TCP v4 | TCP v6 | UDP v4 | UDP v6 | > +---------------------+---------+---------+---------+---------+ > | ZCopy (MB) | 8842 | 8735 | 10072 | 9380 | > +---------------------+---------+---------+---------+---------+ > | New ZCopy (MB) | 9366 | 9477 | 10108 | 9385 | > +---------------------+---------+---------+---------+---------+ > | New ZCopy / ZCopy | 106.00% | 108.28% | 100.31% | 100.01% | > +---------------------+---------+---------+---------+---------+ > > In conclusion, when notification interval is small or notifications are > hard to be coalesced, the new mechanism is highly recommended. Otherwise, > the performance gain from the new mechanism is very limited. > > Signed-off-by: Zijian Zhang <[email protected]> > Signed-off-by: Xiaochun Lu <[email protected]> > --- > tools/testing/selftests/net/msg_zerocopy.c | 111 ++++++++++++++++++-- > tools/testing/selftests/net/msg_zerocopy.sh | 1 + > 2 files changed, 105 insertions(+), 7 deletions(-) > > diff --git a/tools/testing/selftests/net/msg_zerocopy.c > b/tools/testing/selftests/net/msg_zerocopy.c ... > @@ -466,6 +504,44 @@ static void do_recv_completions(int fd, int domain) > sends_since_notify = 0; > } > > +static void do_recv_completions2(void) > +{ > + struct cmsghdr *cm = (struct cmsghdr *)zc_ckbuf; > + struct zc_info *zc_info; > + __u32 hi, lo, range; > + __u8 zerocopy; > + int i; > + > + zc_info = (struct zc_info *)CMSG_DATA(cm); > + for (i = 0; i < zc_info->size; i++) { > + hi = zc_info->arr[i].hi; > + lo = zc_info->arr[i].lo; > + zerocopy = zc_info->arr[i].zerocopy; > + range = hi - lo + 1; > + > + if (cfg_verbose && lo != next_completion) > + fprintf(stderr, "gap: %u..%u does not append to %u\n", > + lo, hi, next_completion); > + next_completion = hi + 1; > + > + if (zerocopied == -1) > + zerocopied = zerocopy; > + else if (zerocopied != zerocopy) { > + fprintf(stderr, "serr: inconsistent\n"); > + zerocopied = zerocopy; > + } nit: If any arms of a conditional have {}, then all arms should have them > + > + completions += range; > + > + if (cfg_verbose >= 2) > + fprintf(stderr, "completed: %u (h=%u l=%u)\n", > + range, hi, lo); > + } > + > + sends_since_notify = 0; > + added_zcopy_info = false; > +} ...
