On Tue, 9 Nov 2021 08:03:18 -0500, Michael S. Tsirkin <m...@redhat.com> wrote: > On Mon, Nov 08, 2021 at 07:49:48PM +0800, Xuan Zhuo wrote: > > If the VIRTIO_RING_F_INDIRECT_DESC negotiation succeeds, and the number > > of sgs used for sending packets is greater than 1. We must constantly > > call __kmalloc/kfree to allocate/release desc. > > > > In the case of extremely fast package delivery, the overhead cannot be > > ignored: > > > > 27.46% [kernel] [k] virtqueue_add > > 16.66% [kernel] [k] detach_buf_split > > 16.51% [kernel] [k] virtnet_xsk_xmit > > 14.04% [kernel] [k] virtqueue_add_outbuf > > 5.18% [kernel] [k] __kmalloc > > 4.08% [kernel] [k] kfree > > 2.80% [kernel] [k] virtqueue_get_buf_ctx > > 2.22% [kernel] [k] xsk_tx_peek_desc > > 2.08% [kernel] [k] memset_erms > > 0.83% [kernel] [k] virtqueue_kick_prepare > > 0.76% [kernel] [k] virtnet_xsk_run > > 0.62% [kernel] [k] __free_old_xmit_ptr > > 0.60% [kernel] [k] vring_map_one_sg > > 0.53% [kernel] [k] native_apic_mem_write > > 0.46% [kernel] [k] sg_next > > 0.43% [kernel] [k] sg_init_table > > 0.41% [kernel] [k] kmalloc_slab > > > > This patch adds a cache function to virtio to cache these allocated indirect > > desc instead of constantly allocating and releasing desc. > > > > v4: > > 1. Only allow desc cache when VIRTIO_RING_F_INDIRECT_DESC negotiation > > is successful > > 2. The desc cache threshold can be set for each virtqueue > > > > v3: > > pre-allocate per buffer indirect descriptors array > > So I'm not sure why we are doing that. Did it improve anything?
1. Don't call kmalloc in the data path, this is not the point 2. Allocate memory on the cpu of the initialization device to ensure that the indirect desc cache and vq desc are on the same numa. Thanks. > > > > v2: > > use struct list_head to cache the desc > > > > Xuan Zhuo (3): > > virtio: cache indirect desc for split > > virtio: cache indirect desc for packed > > virtio-net: enable virtio desc cache > > > > drivers/net/virtio_net.c | 12 ++- > > drivers/virtio/virtio_ring.c | 152 +++++++++++++++++++++++++++++++---- > > include/linux/virtio.h | 17 ++++ > > 3 files changed, 163 insertions(+), 18 deletions(-) > > > > -- > > 2.31.0 > _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
