On 3/6/2024 8:21 PM, Stephen Hemminger wrote:
> On Wed, 6 Mar 2024 16:14:51 +0000
> Ferruh Yigit <ferruh.yi...@amd.com> wrote:
>
>> On 2/29/2024 5:56 PM, Stephen Hemminger wrote:
>>> The tap device needs to exchange file descriptors for tx and rx.
>>> But the EAL MP layer has limit of 8 file descriptors per message.
>>> The ideal resolution would be to increase the number of file
>>> descriptors allowed for rte_mp_sendmsg(), but this would break
>>> the ABI. Workaround the constraint by breaking into multiple messages.
>>>
>>> Do not hide errors about MP message failures.
>>>
>>> Signed-off-by: Stephen Hemminger <step...@networkplumber.org>
>>> ---
>>> drivers/net/tap/rte_eth_tap.c | 40 +++++++++++++++++++++++++++++------
>>> 1 file changed, 33 insertions(+), 7 deletions(-)
>>>
>>> diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c
>>> index 69d9da695bed..df18c328f498 100644
>>> --- a/drivers/net/tap/rte_eth_tap.c
>>> +++ b/drivers/net/tap/rte_eth_tap.c
>>> @@ -863,21 +863,44 @@ tap_mp_req_on_rxtx(struct rte_eth_dev *dev)
>>> msg.fds[fd_iterator++] = process_private->txq_fds[i];
>>> msg.num_fds++;
>>> request_param->txq_count++;
>>> +
>>> + /* Need to break request into chunks */
>>> + if (fd_iterator >= RTE_MP_MAX_FD_NUM) {
>>> + err = rte_mp_sendmsg(&msg);
>>> + if (err < 0)
>>> + goto fail;
>>> +
>>> + fd_iterator = 0;
>>> + msg.num_fds = 0;
>>> + request_param->txq_count = 0;
>>> + }
>>> }
>>> for (i = 0; i < dev->data->nb_rx_queues; i++) {
>>> msg.fds[fd_iterator++] = process_private->rxq_fds[i];
>>> msg.num_fds++;
>>> request_param->rxq_count++;
>>> +
>>> + if (fd_iterator >= RTE_MP_MAX_FD_NUM) {
>>> + err = rte_mp_sendmsg(&msg);
>>> + if (err < 0)
>>> + goto fail;
>>> +
>>> + fd_iterator = 0;
>>> + msg.num_fds = 0;
>>> + request_param->rxq_count = 0;
>>> + }
>>> }
>>
>> Hi Stephen,
>>
>> Did you able to verify with more than 4 queues?
>>
>> As far as I can see, in the secondary counterpart of the
>> 'rte_mp_sendmsg()', each time secondary index starts from 0, and
>> subsequent calls overwrites the fds in secondary.
>> So practically still only 4 queues works.
>
> I got 4 queues setup, but looks like they are trash in secondary.
> Probably best to revert this and fix it by bumping RTE_MP_MAX_FD_NUM.
> This is better, but does take some ABI issue handling.
>
We can increase RTE_MP_MAX_FD_NUM but still there will be a limit.
Can't it be possible to update 'rte_mp_sendmsg()' to support multiple
'rte_mp_sendmsg()' calls in this patch?
Also need to check if fds size is less than 'RTE_PMD_TAP_MAX_QUEUES'
with multiple 'rte_mp_sendmsg()' call support.
