[dpdk-dev] rte_eth_tx_queue_setup() failing (error -22) when setting up tx queues on a VMXNET3 port...
2015-11-25 17:55, Montorsi, Francesco: > Since I was using the rte_eth_rx_queue_count() function in one place, and > VMXNET3 does not support it, I'm getting a SEGFAULT. > So next question is: is user's task to check for validity of pointers inside > dev_ops before calling driver functions? Because rte_eth_rx_queue_count() and > companion funcitons have no safety checks apparently (!!!) Bruce already sent a fix for this issue. It should be applied on HEAD in few minutes.
[dpdk-dev] rte_eth_tx_queue_setup() failing (error -22) when setting up tx queues on a VMXNET3 port...
Hi Stephen,
> -Original Message-
> From: Stephen Hemminger [mailto:stephen at networkplumber.org]
>
> If you read the source, you will see that it there are log messages enabled if
> you configure with LIBRTE_ETHDEV_DEBUG enabled and rebuild DPDK. And
> there are log messages for VMXNETE3 controlled by
> LIBRTE_VMXNET3_DEBUG_INIT
Thanks for this hint. I didn't know actually where to read... anyway enabling
these additional debug messages I found the cause of the problem: the minimal
TX queue ring size for VMXNET3 is 512; I was passing 64.
Fixed this problem, I'm now on the next one I discovered that apparently
the driver for VMXNET3 supports only a limited set of functions:
(gdb) p *dev->dev_ops
$3 = {dev_configure = 0x75cda5 , dev_start = 0x75d653
, dev_stop = 0x75d799 , dev_set_link_up =
0, dev_set_link_down = 0,
dev_close = 0x75d8ab , promiscuous_enable = 0x75dd12
, promiscuous_disable = 0x75ddf6
,
allmulticast_enable = 0x75df0c ,
allmulticast_disable = 0x75df40 , link_update
= 0x75dbd9 ,
stats_get = 0x75d8fe , stats_reset = 0, xstats_get =
0, xstats_reset = 0, queue_stats_mapping_set = 0, dev_infos_get = 0x75db7f
,
mtu_set = 0, vlan_filter_set = 0x75df74 ,
vlan_tpid_set = 0, vlan_strip_queue_set = 0, vlan_offload_set = 0x75e455
,
vlan_pvid_set = 0, rx_queue_start = 0, rx_queue_stop = 0, tx_queue_start = 0,
tx_queue_stop = 0, rx_queue_setup = 0x75bfb8 ,
rx_queue_release = 0x74b069 , rx_queue_count =
0, rx_descriptor_done = 0, rx_queue_intr_enable = 0, rx_queue_intr_disable = 0,
tx_queue_setup = 0x75bbc6 , tx_queue_release =
0x74b03c , dev_led_on = 0, dev_led_off = 0,
flow_ctrl_get = 0,
flow_ctrl_set = 0, priority_flow_ctrl_set = 0, mac_addr_remove = 0,
mac_addr_add = 0, mac_addr_set = 0, uc_hash_table_set = 0,
uc_all_hash_table_set = 0, mirror_rule_set = 0,
mirror_rule_reset = 0, set_vf_rx_mode = 0, set_vf_rx = 0, set_vf_tx = 0,
set_vf_vlan_filter = 0, udp_tunnel_add = 0, udp_tunnel_del = 0,
set_queue_rate_limit = 0, set_vf_rate_limit = 0,
fdir_add_signature_filter = 0, fdir_update_signature_filter = 0,
fdir_remove_signature_filter = 0, fdir_infos_get = 0, fdir_add_perfect_filter =
0, fdir_update_perfect_filter = 0,
fdir_remove_perfect_filter = 0, fdir_set_masks = 0, reta_update = 0,
reta_query = 0, get_reg_length = 0, get_reg = 0, get_eeprom_length = 0,
get_eeprom = 0, set_eeprom = 0,
rss_hash_update = 0, rss_hash_conf_get = 0, filter_ctrl = 0, set_mc_addr_list
= 0, timesync_enable = 0, timesync_disable = 0, timesync_read_rx_timestamp = 0,
timesync_read_tx_timestamp = 0}
Since I was using the rte_eth_rx_queue_count() function in one place, and
VMXNET3 does not support it, I'm getting a SEGFAULT.
So next question is: is user's task to check for validity of pointers inside
dev_ops before calling driver functions? Because rte_eth_rx_queue_count() and
companion funcitons have no safety checks apparently (!!!)
Thanks!
Francesco
[dpdk-dev] rte_eth_tx_queue_setup() failing (error -22) when setting up tx queues on a VMXNET3 port...
Hi all,
I have a server running VMWare ESXi 5.5.0 and VMWare vCenter 5.5.0. On such
server I created a VM with two VMXNET3 NIC cards (one for management, the other
one should be used with DPDK to enable fast-RX of packets coming from other VMs
/ bare-metal NICs).
Inside the VM I have successfully created 2MB hugepages, loaded igb_uio driver
and binded the 2nd VMXNET3 NIC to igb_uio driver:
--
$ ./dpdk_nic_bind.py --status
Network devices using DPDK-compatible driver
:0b:00.0 'VMXNET3 Ethernet Controller' drv=igb_uio unused=vmxnet3
Network devices using kernel driver
===
:03:00.0 'VMXNET3 Ethernet Controller' if=eth0 drv=vmxnet3 unused=igb_uio
*Active*
Other network devices
=
--
However when I start my DPDK-application I get the following error during TX
queue initialization:
ERR rte_eth_tx_queue_setup: err=-22, port=0: Unknown error -22
This is the code that I'm using:
...
rte_eth_dev_info_get(m_portid, m_dev_info);
// proceed with configuration
struct rte_eth_conf port_conf;
memset(_conf, 0, sizeof(port_conf));
port_conf.rxmode.mq_mode = ETH_MQ_RX_RSS;
/** The multi-queue packet distribution mode to be used, e.g.
RSS.; this is important to use multiple RX queues per port ID */
port_conf.rxmode.max_rx_pkt_len = ETHER_MAX_LEN;
port_conf.rxmode.hw_strip_crc = 1; /**< enable CRC stripping by
hardware */
port_conf.rxmode.jumbo_frame= 1; /**< Jumbo Frame Support enabled */
//port_conf.rxmode.enable_lro = 1; /**< Enable LRO */
// NOT SUPPORTED ON TESTED HW
port_conf.rx_adv_conf.rss_conf.rss_hf = ETH_RSS_IP;
port_conf.txmode.mq_mode = ETH_MQ_TX_NONE;
m_num_queues = MIN(64, m_dev_info->max_rx_queues);
m_num_queues = MIN(128, m_num_queues);
ret = rte_eth_dev_configure(m_portid, m_num_queues, 1 /* number of tx
queues */, _conf);
if (ret < 0)
{
HMLogError("HwEmulDPDKPort::init() rte_eth_dev_configure:
err=%d, port=%u: %s", ret, m_portid, rte_strerror(ret));
return false;
}
// init one TX queue: even if we never ever TX packets, at least 1
queue is needed!
--->ret = rte_eth_tx_queue_setup(m_portid, 0 /* queue ID */, 64 /* num
descriptors */, rte_eth_dev_socket_id(m_portid), NULL);
if (ret < 0)
{
// retry with just 1 descriptor
--->ret = rte_eth_tx_queue_setup(m_portid, 0 /* queue ID */, 1 /*
num descriptors */, rte_eth_dev_socket_id(m_portid), NULL);
if (ret < 0)
{
HMLogError("HwEmulDPDKPort::init()
rte_eth_tx_queue_setup: err=%d, port=%u: %s", ret, m_portid, rte_strerror(ret));
return false;
}
}
Basically since I want to only receive packets (no TX) I'm not really
interested in TX queues / num of TX descriptors... However I have troubles
decrypting this -22 error code... any hint?
Thanks a lot,
Francesco Montorsi
[dpdk-dev] rte_eth_tx_queue_setup() failing (error -22) when setting up tx queues on a VMXNET3 port...
On Wed, 25 Nov 2015 10:32:33 +
"Montorsi, Francesco" wrote:
> Hi all,
> I have a server running VMWare ESXi 5.5.0 and VMWare vCenter 5.5.0. On such
> server I created a VM with two VMXNET3 NIC cards (one for management, the
> other one should be used with DPDK to enable fast-RX of packets coming from
> other VMs / bare-metal NICs).
>
> Inside the VM I have successfully created 2MB hugepages, loaded igb_uio
> driver and binded the 2nd VMXNET3 NIC to igb_uio driver:
>
> --
> $ ./dpdk_nic_bind.py --status
>
> Network devices using DPDK-compatible driver
>
> :0b:00.0 'VMXNET3 Ethernet Controller' drv=igb_uio unused=vmxnet3
>
> Network devices using kernel driver
> ===
> :03:00.0 'VMXNET3 Ethernet Controller' if=eth0 drv=vmxnet3 unused=igb_uio
> *Active*
>
> Other network devices
> =
>
> --
>
> However when I start my DPDK-application I get the following error during TX
> queue initialization:
>
> ERR rte_eth_tx_queue_setup: err=-22, port=0: Unknown error -22
>
> This is the code that I'm using:
>
> ...
>rte_eth_dev_info_get(m_portid, m_dev_info);
>
>
> // proceed with configuration
>
> struct rte_eth_conf port_conf;
> memset(_conf, 0, sizeof(port_conf));
>
> port_conf.rxmode.mq_mode = ETH_MQ_RX_RSS;
> /** The multi-queue packet distribution mode to be used, e.g.
> RSS.; this is important to use multiple RX queues per port ID */
> port_conf.rxmode.max_rx_pkt_len = ETHER_MAX_LEN;
> port_conf.rxmode.hw_strip_crc = 1; /**< enable CRC stripping by
> hardware */
> port_conf.rxmode.jumbo_frame= 1; /**< Jumbo Frame Support enabled */
> //port_conf.rxmode.enable_lro = 1; /**< Enable LRO */
> // NOT SUPPORTED ON TESTED HW
> port_conf.rx_adv_conf.rss_conf.rss_hf = ETH_RSS_IP;
> port_conf.txmode.mq_mode = ETH_MQ_TX_NONE;
>
> m_num_queues = MIN(64, m_dev_info->max_rx_queues);
> m_num_queues = MIN(128, m_num_queues);
> ret = rte_eth_dev_configure(m_portid, m_num_queues, 1 /* number of tx
> queues */, _conf);
> if (ret < 0)
> {
> HMLogError("HwEmulDPDKPort::init() rte_eth_dev_configure:
> err=%d, port=%u: %s", ret, m_portid, rte_strerror(ret));
> return false;
> }
>
> // init one TX queue: even if we never ever TX packets, at least 1
> queue is needed!
> ---> ret = rte_eth_tx_queue_setup(m_portid, 0 /* queue ID */, 64 /* num
> descriptors */, rte_eth_dev_socket_id(m_portid), NULL);
> if (ret < 0)
> {
> // retry with just 1 descriptor
>
> ---> ret = rte_eth_tx_queue_setup(m_portid, 0 /* queue ID */, 1 /*
> num descriptors */, rte_eth_dev_socket_id(m_portid), NULL);
> if (ret < 0)
> {
> HMLogError("HwEmulDPDKPort::init()
> rte_eth_tx_queue_setup: err=%d, port=%u: %s", ret, m_portid,
> rte_strerror(ret));
> return false;
> }
> }
>
>
> Basically since I want to only receive packets (no TX) I'm not really
> interested in TX queues / num of TX descriptors... However I have troubles
> decrypting this -22 error code... any hint?
First advice, DPDK is open source, therefore you have the source use it.
Error codes match Linux/Unix errno's. You can use strerror(-ret) to find the
value
or look in errno.h. Hint -22 == EINVAL
If you read the source, you will see that it there are log messages enabled if
you
configure with LIBRTE_ETHDEV_DEBUG enabled and rebuild DPDK. And there are log
messages
for VMXNETE3 controlled by LIBRTE_VMXNET3_DEBUG_INIT
