On Tue, 3 Nov 2026 15:14:56 +0100, Przemek Kitszel wrote:
> Add support for up to 256 VF queues. To enable more than usual 16, user
> needs to assign GLOBAL LUT (devlink resource named "rss/lut_512") for 64
> queues or PF LUT (name "rss/lut_2048") for maximum of 256 queues.
I ran the whole v4 on an E810-C for SFP (4 ports, 10G DAC between two
of them, FW 8.1.1 with mgmt API 1.7.11, DDP COMMS 1.3.55.0) on top of
net-next 7b6ab4dbc156 with lockdep, and again with KASAN. The
documented flows work as described: PF and VF LUT assignment, 64 and
256 queues on a VF (also with 65 MSI-X), VF FLR, iavf unbind and bind,
PFR, CORER and GLOBR restoring the default assignment, the ADQ
exclusion in both directions, 16 VFs taking all 16 global LUTs, and
teardown with LUTs still assigned. The two problems below come with
this patch.
> diff --git a/drivers/net/ethernet/intel/ice/virt/virtchnl.c
> b/drivers/net/ethernet/intel/ice/virt/virtchnl.c
[...]
> @@ -329,12 +328,16 @@ static int ice_vc_get_vf_res_msg(struct ice_vf *vf, u8
> *msg)
> if (vf->driver_caps & VIRTCHNL_VF_CAP_PTP)
> vfres->vf_cap_flags |= VIRTCHNL_VF_CAP_PTP;
>
> + if (vf->driver_caps & VIRTCHNL_VF_LARGE_NUM_QPAIRS &&
> + vsi->rss_lut_type != ICE_LUT_VSI)
> + vfres->vf_cap_flags |= VIRTCHNL_VF_LARGE_NUM_QPAIRS;
> +
> vfres->num_vsis = 1;
> /* Tx and Rx queue are equal for VF */
> vfres->num_queue_pairs = vsi->num_txq;
> vfres->max_vectors = vf->num_msix;
> vfres->rss_key_size = ICE_VSIQF_HKEY_ARRAY_SIZE;
> - vfres->rss_lut_size = ICE_LUT_VSI_SIZE;
> + vfres->rss_lut_size = vsi->rss_table_size;
This reports the 512 or 2048 entry LUT to every VF, including VFs that
never negotiated VIRTCHNL_VF_LARGE_NUM_QPAIRS. The iavf before this
series copies the new rss_lut_size in iavf_parse_vf_resource_msg()
when the GET_VF_RESOURCES reply of the reset arrives, but its rss_lut
buffer was allocated once in iavf_init_config_adapter() with the 64
entries reported back then. From that point iavf_get_rxfh() and
iavf_set_rss_lut() read rss_lut_size bytes out of the 64-byte object,
so the LUT sent to the PF carries neighbouring heap memory, and the
next VF reset writes rss_lut_size bytes into it through
iavf_reset_step() -> iavf_init_rss() -> iavf_fill_rss_lut(). The
in-tree iavf of every released kernel behaves this way, so a guest with
an older kernel hits it as soon as the host admin assigns a LUT, and
presumably other VF drivers that size their LUT once are affected the
same way.
Reproduction on the PF with this series and an in-tree iavf built from
the parent commit bound to the VF:
echo 1 > /sys/bus/pci/devices/0000:04:00.0/sriov_numvfs
devlink resource set pci/0000:04:01.0 path rss/lut_512 size 1
After the deferred VF reset, ethtool -x on the VF showed a 512 entry
table with 60 distinct queue indices for a VF that has 16 queues, two
following ethtool -L calls on the VF (two more VF resets) went through,
and the host hung hard right after that, without reaching the serial
console.
I think the PF has to keep reporting ICE_LUT_VSI_SIZE and leave the
VSI on its VSI LUT for VFs without VIRTCHNL_VF_LARGE_NUM_QPAIRS, or
reject the devlink change for such a VF with an extack message. The
mix of an old VF driver and a new PF driver is exactly what stable
backports and VMs produce.
> diff --git a/drivers/net/ethernet/intel/ice/virt/queues.c
> b/drivers/net/ethernet/intel/ice/virt/queues.c
[...]
> @@ -1005,19 +1005,37 @@ int ice_vc_request_qs_msg(struct ice_vf *vf, u8 *msg)
[...]
> cur_queues = vf->num_vf_qs;
> + /* without LARGE_NUM_QPAIRS, VF can only use the v1 queue opcodes */
> + if (vf->driver_caps & VIRTCHNL_VF_LARGE_NUM_QPAIRS)
> + max_lut_queues = ice_lut_type_to_qs_num(vsi->rss_lut_type);
> + else
> + max_lut_queues = ICE_MAX_QS_PER_VF_VCV1;
> tx_rx_queue_left = min_t(u16, ice_get_avail_txq_count(pf),
> ice_get_avail_rxq_count(pf));
> max_allowed_vf_queues = tx_rx_queue_left + cur_queues;
> + if (req_queues > max_lut_queues) {
> + vfres->num_queue_pairs = max_lut_queues;
> + goto error_param;
> + }
Up to now a VF asks for at most 16 queues, so the ranges VFs free and
take again have the same small size and __ice_vsi_get_qs() finds a
contiguous range for a VF VSI in practice. With requests of 64 and
256 the PF queue space fragments quickly, the ICE_VSI_MAP_SCATTER
fallback kicks in for VF VSIs, and ice_ena_vf_q_mappings() only logs
"Scattered mode for VF Tx queues is not yet implemented" and leaves
VPLAN_TX_QBASE and VPLAN_RX_QBASE unprogrammed for the new range. The
reset still completes and the VF is told it owns the queues.
Reproduction with 16 VFs on one PF:
for vf in 0000:04:01.{0..7} 0000:04:02.{0..7}; do
devlink resource set pci/$vf path rss/lut_512 size 1
done
for vf in enp4s0f0v{0..15}; do ethtool -L $vf combined 64; done
All 16 requests were granted and 12 of the VFs came up with scattered
Tx and Rx mappings. The three of those I checked (VF 3, 4 and 10)
have no connectivity at all, ping fails in both directions and no Rx
queue ever counts a packet, while VFs with contiguous mappings pass
2 to 4 Gbit/s in a short iperf. iavf on an affected VF then loops on
"Failed to configure queues, error IAVF_ERR_PARAM" (CONFIG_VSI_QUEUES
is NACKed by ice_vc_cfg_qs_msg() without any message on the PF side)
and on NETDEV WATCHDOG transmit timeouts, 70 of them on VF 3 alone.
About 20 s after the grants the PF also logged "Tx Malicious Driver
Detection events detected on PF 0 VF 13", a VF with a contiguous
mapping; I did not chase whether the stale mappings cause that.
A request that cannot be served with a contiguous range should be
refused here, or clamped to what can be, and ice_vf_rebuild_vsi()
should fail the reset instead of leaving the VF half mapped. Sashiko's
remark about max_lut_queues not being clamped to max_allowed_vf_queues
is about the same spot.
One more data point, not caused by this series. While stressing PFR
together with the LUT changes I hit a slab-use-after-free in
ice_add_rss_cfg_sync() called from ice_replay_rss_cfg(). It reproduces
on plain net-next with "ethtool -X <pf> equal 4" followed by a PFR, so
I will report it separately with the details.
Regards,
Petr
