[dpdk-dev] [PATCH] virtio: fix allocating virtnet_rx not mem aligned
On Sun, Jun 12, 2016 at 02:29:42PM +, Jianfeng Tan wrote:
> Compile DPDK with clang, below line in virtio_rxtx.c could be
> optimized with four "VMOVAPS ymm, m256".
> memset(>fake_mbuf, 0, sizeof(rxvq->fake_mbuf));
>
> This instruction requires memory address is 32-byte aligned.
> Or, it leads to segfault. Although only tested with Clang 3.6.0,
> it can be reproduced in any compilers, which do aggressive
> optimization, aka, change memset of known length to VMOVAPS.
>
> The fact that struct rte_mbuf is cache line aligned, can only make
> sure fake_mbuf is aligned compared to the start address of struct
> virtnet_rx. Unfortunately, this address is not necessarily aligned
> because it's allocated by:
> rxvq = (struct virtnet_rx *)RTE_PTR_ADD(vq, sz_vq);
>
> When sz_vq is not aligned, then rxvq cannot be allocated with an
> aligned address, and then rxvq->fake_mbuf (addr of rxvq + cache line
> size) is not an aligned address.
>
> The fix is very simple that making sz_vq 32-byte aligned. Here we
> make it cache line aligned for future optimization.
>
> Fixes: a900472aedef ("virtio: split virtio Rx/Tx queue")
Folded this fix (and the other fix) to above commit, so that we could
have a clean working tree.
Thanks for good catch!
--yliu
[dpdk-dev] [PATCH] virtio: fix allocating virtnet_rx not mem aligned
On Mon, Jun 13, 2016 at 10:06:22AM +, Tan, Jianfeng wrote: > > > > -Original Message- > > From: Yuanhan Liu [mailto:yuanhan.liu at linux.intel.com] > > Sent: Monday, June 13, 2016 5:52 PM > > To: Tan, Jianfeng > > Cc: dev at dpdk.org; Xie, Huawei > > Subject: Re: [dpdk-dev] [PATCH] virtio: fix allocating virtnet_rx not mem > > aligned > > > > On Mon, Jun 13, 2016 at 05:21:01PM +0800, Yuanhan Liu wrote: > > > On Sun, Jun 12, 2016 at 02:29:42PM +, Jianfeng Tan wrote: > > > > Compile DPDK with clang, below line in virtio_rxtx.c could be > > > > optimized with four "VMOVAPS ymm, m256". > > > > memset(>fake_mbuf, 0, sizeof(rxvq->fake_mbuf)); > > > > > > > > This instruction requires memory address is 32-byte aligned. > > > > Or, it leads to segfault. > > > > > > That looks like a dangerous optimization to me. If that's the case, > > > doesn't it mean we have to make sure the address is always aligned > > > properly while calling memset? > > > > Above is just a side note. Anyway, I think making sure vq is cache > > aligned is good here. So, I will apply it. BTW, do you mind if I > > squash your 2 fixes into Huawei's Rx/Tx split commit? His commit is > > not pushed to upstream yet, therefore I can still do rebase: I'm > > thinking it's better to have one working commit other than one broken > > commit followed with several fixing commits. And of course, I will > > mention your contribution in the commit log. > > Not a problem from my side. But Huawei seems to have concerns on this fix. > You can do that for the other fix firstly. What's the concern? Despite the clang issue (I still have no idea why clang would go that aggressively), I think making vq cache aligned is generically a good idea. --yliu
[dpdk-dev] [PATCH] virtio: fix allocating virtnet_rx not mem aligned
On 6/13/2016 5:21 PM, Yuanhan Liu wrote: > On Sun, Jun 12, 2016 at 02:29:42PM +, Jianfeng Tan wrote: >> Compile DPDK with clang, below line in virtio_rxtx.c could be >> optimized with four "VMOVAPS ymm, m256". >>memset(>fake_mbuf, 0, sizeof(rxvq->fake_mbuf)); >> >> This instruction requires memory address is 32-byte aligned. >> Or, it leads to segfault. > That looks like a dangerous optimization to me.If that's the case, > doesn't it mean we have to make sure the address is always aligned > properly while calling memset? I guess clang does such optimization when length is a 32-byte aligned immediate number. May need more information here. Thanks, Jianfeng > > --yliu
[dpdk-dev] [PATCH] virtio: fix allocating virtnet_rx not mem aligned
On Mon, Jun 13, 2016 at 05:21:01PM +0800, Yuanhan Liu wrote: > On Sun, Jun 12, 2016 at 02:29:42PM +, Jianfeng Tan wrote: > > Compile DPDK with clang, below line in virtio_rxtx.c could be > > optimized with four "VMOVAPS ymm, m256". > > memset(>fake_mbuf, 0, sizeof(rxvq->fake_mbuf)); > > > > This instruction requires memory address is 32-byte aligned. > > Or, it leads to segfault. > > That looks like a dangerous optimization to me. If that's the case, > doesn't it mean we have to make sure the address is always aligned > properly while calling memset? Above is just a side note. Anyway, I think making sure vq is cache aligned is good here. So, I will apply it. BTW, do you mind if I squash your 2 fixes into Huawei's Rx/Tx split commit? His commit is not pushed to upstream yet, therefore I can still do rebase: I'm thinking it's better to have one working commit other than one broken commit followed with several fixing commits. And of course, I will mention your contribution in the commit log. --yliu
[dpdk-dev] [PATCH] virtio: fix allocating virtnet_rx not mem aligned
On Sun, Jun 12, 2016 at 02:29:42PM +, Jianfeng Tan wrote: > Compile DPDK with clang, below line in virtio_rxtx.c could be > optimized with four "VMOVAPS ymm, m256". > memset(>fake_mbuf, 0, sizeof(rxvq->fake_mbuf)); > > This instruction requires memory address is 32-byte aligned. > Or, it leads to segfault. That looks like a dangerous optimization to me. If that's the case, doesn't it mean we have to make sure the address is always aligned properly while calling memset? --yliu
[dpdk-dev] [PATCH] virtio: fix allocating virtnet_rx not mem aligned
> -Original Message- > From: Yuanhan Liu [mailto:yuanhan.liu at linux.intel.com] > Sent: Monday, June 13, 2016 5:52 PM > To: Tan, Jianfeng > Cc: dev at dpdk.org; Xie, Huawei > Subject: Re: [dpdk-dev] [PATCH] virtio: fix allocating virtnet_rx not mem > aligned > > On Mon, Jun 13, 2016 at 05:21:01PM +0800, Yuanhan Liu wrote: > > On Sun, Jun 12, 2016 at 02:29:42PM +, Jianfeng Tan wrote: > > > Compile DPDK with clang, below line in virtio_rxtx.c could be > > > optimized with four "VMOVAPS ymm, m256". > > > memset(>fake_mbuf, 0, sizeof(rxvq->fake_mbuf)); > > > > > > This instruction requires memory address is 32-byte aligned. > > > Or, it leads to segfault. > > > > That looks like a dangerous optimization to me. If that's the case, > > doesn't it mean we have to make sure the address is always aligned > > properly while calling memset? > > Above is just a side note. Anyway, I think making sure vq is cache > aligned is good here. So, I will apply it. BTW, do you mind if I > squash your 2 fixes into Huawei's Rx/Tx split commit? His commit is > not pushed to upstream yet, therefore I can still do rebase: I'm > thinking it's better to have one working commit other than one broken > commit followed with several fixing commits. And of course, I will > mention your contribution in the commit log. Not a problem from my side. But Huawei seems to have concerns on this fix. You can do that for the other fix firstly. Thanks, Jianfeng > > --yliu
[dpdk-dev] [PATCH] virtio: fix allocating virtnet_rx not mem aligned
Compile DPDK with clang, below line in virtio_rxtx.c could be
optimized with four "VMOVAPS ymm, m256".
memset(>fake_mbuf, 0, sizeof(rxvq->fake_mbuf));
This instruction requires memory address is 32-byte aligned.
Or, it leads to segfault. Although only tested with Clang 3.6.0,
it can be reproduced in any compilers, which do aggressive
optimization, aka, change memset of known length to VMOVAPS.
The fact that struct rte_mbuf is cache line aligned, can only make
sure fake_mbuf is aligned compared to the start address of struct
virtnet_rx. Unfortunately, this address is not necessarily aligned
because it's allocated by:
rxvq = (struct virtnet_rx *)RTE_PTR_ADD(vq, sz_vq);
When sz_vq is not aligned, then rxvq cannot be allocated with an
aligned address, and then rxvq->fake_mbuf (addr of rxvq + cache line
size) is not an aligned address.
The fix is very simple that making sz_vq 32-byte aligned. Here we
make it cache line aligned for future optimization.
Fixes: a900472aedef ("virtio: split virtio Rx/Tx queue")
Signed-off-by: Jianfeng Tan
---
drivers/net/virtio/virtio_ethdev.c | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/net/virtio/virtio_ethdev.c
b/drivers/net/virtio/virtio_ethdev.c
index a995520..ad0f5a6 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -337,7 +337,10 @@ int virtio_dev_queue_setup(struct rte_eth_dev *dev,
snprintf(vq_name, sizeof(vq_name), "port%d_%s%d",
dev->data->port_id, queue_names[queue_type], queue_idx);
- sz_vq = sizeof(*vq) + vq_size * sizeof(struct vq_desc_extra);
+
+ sz_vq = RTE_ALIGN_CEIL(sizeof(*vq) +
+ vq_size * sizeof(struct vq_desc_extra),
+ RTE_CACHE_LINE_SIZE);
if (queue_type == VTNET_RQ) {
sz_q = sz_vq + sizeof(*rxvq);
} else if (queue_type == VTNET_TQ) {
--
2.1.4
