On 4/6/2016 1:44 PM, Yuanhan Liu wrote:
> On Wed, Apr 06, 2016 at 01:32:12PM +0800, Tan, Jianfeng wrote:
>> Hi,
>>
>> Just out of interest, seems that the message handling thread which runs
>> new_device() is pthread_create() from the thread which calls the
>> dev_start(), usually master thread, right? But it's not necessary to be the
>> master thread to poll pkts from this vhost port, right? So what's the
>> significance to record the numa_node information of message handling thread
>> here? Shall we make the decision of numa_realloc based on the final PMD
>> thread who is responsible for polling this vhost port?
> It doesn't matter on which core we made the decision: the result
> would be same since we are querying the numa node info of the
> virtio_net dev struct.
OK, according to your hint, read numa_realloc() again, it's to keep *dev
(struct virtio_net), *dev->virtqueue[], on the same numa socket of
shared memory with virtio?
And numa_realloc() is called in vhost_set_vring_addr(), which is after
new_device()?
>
> --yliu
>> It's not related to this patch itself. And it seems good to me.
>>
>>
>> Thanks,
>> Jianfeng
>>
>>
>>
>> On 4/6/2016 12:09 AM, Ciara Loftus wrote:
>>> After some testing, it was found that retrieving numa information
>>> about a vhost device via a call to get_mempolicy is more
>>> accurate when performed during the new_device callback versus
>>> the vring_state_changed callback, in particular upon initial boot
>>> of the VM. Performing this check during new_device is also
>>> potentially more efficient as this callback is only triggered once
>>> during device initialisation, compared with vring_state_changed
>>> which may be called multiple times depending on the number of
>>> queues assigned to the device.
>>>
>>> Reorganise the code to perform this check and assign the correct
>>> socket_id to the device during the new_device callback.
>>>
>>> Signed-off-by: Ciara Loftus <ciara.loftus at intel.com>
>>> ---
>>> drivers/net/vhost/rte_eth_vhost.c | 28 ++++++++++++++--------------
>>> 1 file changed, 14 insertions(+), 14 deletions(-)
>>>
>>> diff --git a/drivers/net/vhost/rte_eth_vhost.c
>>> b/drivers/net/vhost/rte_eth_vhost.c
>>> index 4cc6bec..b1eb082 100644
>>> --- a/drivers/net/vhost/rte_eth_vhost.c
>>> +++ b/drivers/net/vhost/rte_eth_vhost.c
>>> @@ -229,6 +229,9 @@ new_device(struct virtio_net *dev)
>>> struct pmd_internal *internal;
>>> struct vhost_queue *vq;
>>> unsigned i;
>>> +#ifdef RTE_LIBRTE_VHOST_NUMA
>>> + int newnode, ret;
>>> +#endif
>>> if (dev == NULL) {
>>> RTE_LOG(INFO, PMD, "Invalid argument\n");
>>> @@ -244,6 +247,17 @@ new_device(struct virtio_net *dev)
>>> eth_dev = list->eth_dev;
>>> internal = eth_dev->data->dev_private;
>>> +#ifdef RTE_LIBRTE_VHOST_NUMA
>>> + ret = get_mempolicy(&newnode, NULL, 0, dev,
>>> + MPOL_F_NODE | MPOL_F_ADDR);
>>> + if (ret < 0) {
>>> + RTE_LOG(ERR, PMD, "Unknown numa node\n");
>>> + return -1;
>>> + }
>>> +
>>> + eth_dev->data->numa_node = newnode;
If it's correct of above analysis, dev, here, could be numa_realloc() later?
Thanks,
Jianfeng
>>> +#endif
>>> +
>>> for (i = 0; i < eth_dev->data->nb_rx_queues; i++) {
>>> vq = eth_dev->data->rx_queues[i];
>>> if (vq == NULL)
>>> @@ -352,9 +366,6 @@ vring_state_changed(struct virtio_net *dev, uint16_t
>>> vring, int enable)
>>> struct rte_vhost_vring_state *state;
>>> struct rte_eth_dev *eth_dev;
>>> struct internal_list *list;
>>> -#ifdef RTE_LIBRTE_VHOST_NUMA
>>> - int newnode, ret;
>>> -#endif
>>> if (dev == NULL) {
>>> RTE_LOG(ERR, PMD, "Invalid argument\n");
>>> @@ -370,17 +381,6 @@ vring_state_changed(struct virtio_net *dev, uint16_t
>>> vring, int enable)
>>> eth_dev = list->eth_dev;
>>> /* won't be NULL */
>>> state = vring_states[eth_dev->data->port_id];
>>> -
>>> -#ifdef RTE_LIBRTE_VHOST_NUMA
>>> - ret = get_mempolicy(&newnode, NULL, 0, dev,
>>> - MPOL_F_NODE | MPOL_F_ADDR);
>>> - if (ret < 0) {
>>> - RTE_LOG(ERR, PMD, "Unknown numa node\n");
>>> - return -1;
>>> - }
>>> -
>>> - eth_dev->data->numa_node = newnode;
>>> -#endif
>>> rte_spinlock_lock(&state->lock);
>>> state->cur[vring] = enable;
>>> state->max_vring = RTE_MAX(vring, state->max_vring);
