Hi all/Maxime/Chenbox, I’m observing an issue with virtio_user (with vhost-net backend) where VLAN offload doesn't seem to insert the VLAN tag, even though both vlan_tci and ol_flags are set correctly in the mbuf. We are using DPDK Version: 24.11 (custom build) and virtio_user for exception path traffic.
Context
-
*Scenario*: When a packet with vlan_tci set is transmitted via
virtio_user with RTE_ETH_TX_OFFLOAD_VLAN_INSERT enabled, the packet
received on the parent TAP interface lacks the VLAN tag. It subsequently
gets dropped, since it was meant for the TAP sub-interface corresponding to
the VLAN.
-
*Expected*: The packet should carry the VLAN tag when seen on the TAP
interface, allowing it to be delivered to the correct sub-interface.
Observations
-
At the point of calling virtio_xmit_pkts():
-
vlan_tci is correctly set (e.g., 61).
-
ol_flags contains RTE_MBUF_F_TX_VLAN (bit 0) => value = 0x1.
-
We confirmed the following in GDB just before calling
virtqueue_enqueue_xmit():
(gdb) p **tx_pkts
$6 = { ..., vlan_tci = 61, ..., ol_flags = 65, ... }
(gdb) p /t tx_pkts.ol_flags
$8 = 1000001
The value 65 (0b1000001) confirms that RTE_ETH_TX_OFFLOAD_VLAN_INSERT (bit
0) is set.
Despite this, the packet captured on the TAP interface is a plain Ethernet
frame, with no VLAN tag.
Adding few more outputs more gdb
(gdb) p rte_eth_devices[33]
$4 = {rx_pkt_burst = 0x558180950c36 <virtio_recv_mergeable_pkts>,
tx_pkt_burst = 0x558180952440 <virtio_xmit_pkts>, tx_pkt_prepare =
0x558180951fa3 <virtio_xmit_pkts_prepare>, rx_queue_count = 0x0,
rx_descriptor_status = 0x0,
tx_queue_count = 0x0, tx_descriptor_status = 0x0, recycle_tx_mbufs_reuse
= 0x0, recycle_rx_descriptors_refill = 0x0, data = 0x60005fd8e080,
process_private = 0x0, dev_ops = 0x55818211e2a0 <virtio_eth_dev_ops>,
flow_fp_ops = 0x55818218d140 <rte_flow_fp_default_ops>, device =
0x558185a960b0, intr_handle = 0x558185a96160, link_intr_cbs = {tqh_first =
0x0, tqh_last = 0x5581825a42f8 <rte_eth_devices+547128>}, post_rx_burst_cbs
= {
0x0 <repeats 1024 times>}, pre_tx_burst_cbs = {0x0 <repeats 1024
times>}, state = RTE_ETH_DEV_ATTACHED, security_ctx = 0x0}
(gdb) p *rte_eth_devices[33].data
$6 = {name = "virtio_user0", '\000' <repeats 51 times>, rx_queues =
0x60005fc2ef80, tx_queues = 0x60005fc2cf00, nb_rx_queues = 1, nb_tx_queues
= 1, sriov = {active = 0 '\000', nb_q_per_pool = 0 '\000', def_vmdq_idx =
0, def_pool_q_idx = 0},
dev_private = 0x60005fc60380, dev_link = {{val64 = 34359738367,
{link_speed = 4294967295, link_duplex = 1, link_autoneg = 1, link_status =
1}}}, dev_conf = {link_speeds = 0, rxmode = {mq_mode = RTE_ETH_MQ_RX_NONE,
mtu = 9000,
max_lro_pkt_size = 0, offloads = 8193, reserved_64s = {0, 0},
reserved_ptrs = {0x0, 0x0}}, txmode = {mq_mode = RTE_ETH_MQ_TX_NONE,
offloads = 32813, pvid = 0, hw_vlan_reject_tagged = 0 '\000',
hw_vlan_reject_untagged = 0 '\000',
hw_vlan_insert_pvid = 0 '\000', reserved_64s = {0, 0}, reserved_ptrs
= {0x0, 0x0}}, lpbk_mode = 0, rx_adv_conf = {rss_conf = {rss_key = 0x0,
rss_key_len = 0 '\000', rss_hf = 0, algorithm =
RTE_ETH_HASH_FUNCTION_DEFAULT},
vmdq_dcb_conf = {nb_queue_pools = 0, enable_default_pool = 0 '\000',
default_pool = 0 '\000', nb_pool_maps = 0 '\000', pool_map = {{vlan_id = 0,
pools = 0} <repeats 64 times>}, dcb_tc = "\000\000\000\000\000\000\000"},
dcb_rx_conf = {
nb_tcs = 0, dcb_tc = "\000\000\000\000\000\000\000"}, vmdq_rx_conf
= {nb_queue_pools = 0, enable_default_pool = 0 '\000', default_pool = 0
'\000', enable_loop_back = 0 '\000', nb_pool_maps = 0 '\000', rx_mode = 0,
pool_map = {{
vlan_id = 0, pools = 0} <repeats 64 times>}}}, tx_adv_conf =
{vmdq_dcb_tx_conf = {nb_queue_pools = 0, dcb_tc =
"\000\000\000\000\000\000\000"}, dcb_tx_conf = {nb_tcs = 0, dcb_tc =
"\000\000\000\000\000\000\000"}, vmdq_tx_conf = {
nb_queue_pools = 0}}, dcb_capability_en = 0, intr_conf = {lsc = 0,
rxq = 0, rmv = 0}}, mtu = 9000, min_rx_buf_size = 4294967295,
rx_mbuf_alloc_failed = 0, mac_addrs = 0x60005ff404c0, mac_pool_sel = {0
<repeats 128 times>},
hash_mac_addrs = 0x0, port_id = 33, promiscuous = 0 '\000', scattered_rx
= 0 '\000', all_multicast = 0 '\000', dev_started = 1 '\001', lro = 0
'\000', dev_configured = 1 '\001', flow_configured = 0 '\000',
rx_queue_state = "\001", '\000' <repeats 1022 times>, tx_queue_state =
"\001", '\000' <repeats 1022 times>, dev_flags = 0, numa_node = -1,
vlan_filter_conf = {ids = {0 <repeats 64 times>}}, owner = {id = 0,
name = '\000' <repeats 63 times>}, representor_id = 0, backer_port_id =
64, flow_ops_mutex = {__data = {__lock = 0, __count = 0, __owner = 0,
__nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev =
0x0, __next = 0x0}},
__size = '\000' <repeats 39 times>, __align = 0}}
(gdb) p *(struct virtio_user_dev*)rte_eth_devices[33].data.dev_private
$3 = {hw = {vqs = 0x60005ff40400, guest_features = 4563441697,
vtnet_hdr_size = 12, started = 1 '\001', weak_barriers = 1 '\001',
vlan_strip = 1 '\001', rx_ol_scatter = true, has_tx_offload = 1 '\001',
has_rx_offload = 0 '\000',
use_vec_rx = 0 '\000', use_vec_tx = 0 '\000', use_inorder_rx = 0
'\000', use_inorder_tx = 0 '\000', opened = 1 '\001', port_id = 33,
mac_addr = "\000PV\235#g", get_speed_via_feat = false, speed = 4294967295,
duplex = 1 '\001',
intr_lsc = 1 '\001', max_mtu = 9698, max_rx_pkt_len = 9030, state_lock
= {locked = 0}, inject_pkts = 0x0, max_queue_pairs = 1, rss_rx_queues = 0,
rss_hash_types = 0, rss_reta = 0x0, rss_key = 0x0,
req_guest_features = 9223372445160806441, cvq = 0x0, use_va = true},
backend_type = VIRTIO_USER_BACKEND_VHOST_KERNEL, is_server = false, callfds
= 0x6000396000c0, kickfds = 0x60005ff40880, mac_specified = 1,
max_queue_pairs = 1,
queue_pairs = 1, queue_size = 2048, features = 4563441697,
device_features = 4563442051, frontend_features = 32, unsupported_features
= 17293822186582074972, status = 15 '\017', net_status = 0, mac_addr =
"\000PV\235#g",
path = "/dev/vhost-net", '\000' <repeats 4081 times>, ifname =
0x558185a04b70 "avi_eth1", vrings = {ptr = 0x60005ff40780, split =
0x60005ff40780, packed = 0x60005ff40780}, packed_queues = 0x0, qp_enabled =
0x60005ff406c0,
ops = 0x55818218cc80 <virtio_ops_kernel>, mutex = {__data = {__lock = 0,
__count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision
= 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39
times>,
__align = 0}, started = true, hw_cvq = false, scvq = 0x0, backend_data
= 0x558185a96100, notify_area = 0x0}
Workaround
If we explicitly insert the VLAN tag into the packet data (i.e., inline
VLAN insertion), the TAP interface sees the tag as expected, and the packet
is correctly forwarded to the intended sub-interface.
Conclusion
It appears that the VLAN offload flag and vlan_tci are not being honored or
processed correctly somewhere in the virtio path with vhost-net backend.
This leads to silent drops on the host kernel side due to missing VLAN tags.
Request
Could you please confirm if this is a known limitation or a bug? If it’s
the latter, I’d be happy to provide more debug info or steps to reproduce.
Thanks,
Samar Yadav
Broadcom
smime.p7s
Description: S/MIME Cryptographic Signature
