On 5/22/2024 4:12 PM, Michal Schmidt wrote:
> irq_set_affinity_hint() is deprecated. Use irq_update_affinity_hint()
> instead. This removes the side-effect of actually applying the affinity.
>
> The driver does not really need to worry about spreading its IRQs across
> CPUs. The core code already takes care of that.
> On the contrary, when the driver applies affinities by itself, it breaks
> the users' expectations:
> 1. The user configures irqbalance with IRQBALANCE_BANNED_CPULIST in
> order to prevent IRQs from being moved to certain CPUs that run a
> real-time workload.
> 2. ice reconfigures VSIs at runtime due to a MIB change
> (ice_dcb_process_lldp_set_mib_change). Reopening a VSI resets the
> affinity in ice_vsi_req_irq_msix().
On an unrelated note, I wonder if this sort of reconfiguration could be
avoided so we don't lose such configuration....
> 3. ice has no idea about irqbalance's config, so it may move an IRQ to
> a banned CPU. The real-time workload suffers unacceptable latency.
>
Given all of these problems, what is remaining for us to completely
remove this API so that future driver authors don't make this mistake again?
> I am not sure if updating the affinity hints is at all useful, because
> irqbalance ignores them since 2016 ([1]), but at least it's harmless.
>
Yea. To be honest, I suspect this sort of code originates from
micro-bench marking with irqbalance disabled. I certainly remember folks
telling me that irqbalance wasn't helpful. In reality I think its
because when doing bench marking or testing you want run-to-run behavior
to stay consistent while irqbalance might be changing and tuning
parameters differently due to other system load.
> This ice change is similar to i40e commit d34c54d1739c ("i40e: Use
> irq_update_affinity_hint()").
>
> [1] https://github.com/Irqbalance/irqbalance/commit/dcc411e7bfdd
>
> Signed-off-by: Michal Schmidt <[email protected]>
> ---
Makes sense.
Reviewed-by: Jacob Keller <[email protected]>
> drivers/net/ethernet/intel/ice/ice_lib.c | 4 ++--
> drivers/net/ethernet/intel/ice/ice_main.c | 4 ++--
> 2 files changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c
> b/drivers/net/ethernet/intel/ice/ice_lib.c
> index 5371e91f6bbb..0f8b622db2b5 100644
> --- a/drivers/net/ethernet/intel/ice/ice_lib.c
> +++ b/drivers/net/ethernet/intel/ice/ice_lib.c
> @@ -2587,8 +2587,8 @@ void ice_vsi_free_irq(struct ice_vsi *vsi)
> if (!IS_ENABLED(CONFIG_RFS_ACCEL))
> irq_set_affinity_notifier(irq_num, NULL);
>
> - /* clear the affinity_mask in the IRQ descriptor */
> - irq_set_affinity_hint(irq_num, NULL);
> + /* clear the affinity_hint in the IRQ descriptor */
> + irq_update_affinity_hint(irq_num, NULL);
> synchronize_irq(irq_num);
> devm_free_irq(ice_pf_to_dev(pf), irq_num, vsi->q_vectors[i]);
> }
> diff --git a/drivers/net/ethernet/intel/ice/ice_main.c
> b/drivers/net/ethernet/intel/ice/ice_main.c
> index f60c022f7960..a5d369b8fed5 100644
> --- a/drivers/net/ethernet/intel/ice/ice_main.c
> +++ b/drivers/net/ethernet/intel/ice/ice_main.c
> @@ -2607,7 +2607,7 @@ static int ice_vsi_req_irq_msix(struct ice_vsi *vsi,
> char *basename)
> }
>
> /* assign the mask for this irq */
> - irq_set_affinity_hint(irq_num, &q_vector->affinity_mask);
> + irq_update_affinity_hint(irq_num, &q_vector->affinity_mask);
> }
>
> err = ice_set_cpu_rx_rmap(vsi);
> @@ -2625,7 +2625,7 @@ static int ice_vsi_req_irq_msix(struct ice_vsi *vsi,
> char *basename)
> irq_num = vsi->q_vectors[vector]->irq.virq;
> if (!IS_ENABLED(CONFIG_RFS_ACCEL))
> irq_set_affinity_notifier(irq_num, NULL);
> - irq_set_affinity_hint(irq_num, NULL);
> + irq_update_affinity_hint(irq_num, NULL);
> devm_free_irq(dev, irq_num, &vsi->q_vectors[vector]);
> }
> return err;
