Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] net: mana: missing TX CQ poll after final sleep in mana_dealloc_queues() - [Medium] net: mana: incomplete dead HWC checks in mana_dealloc_queues() --
commit 4704079fb73cc77fdb1cc7581dd8e498b290c328 Author: Sahil Chandna <[email protected]> net: mana: Avoid waiting on dead HWC after device reset This commit replaces the 120s sleep-based TX drain in mana_dealloc_queues() with synchronous CQ polling and marks the hardware channel dead after a device reset so subsequent HWC commands fail fast instead of waiting for a timeout. > diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c > b/drivers/net/ethernet/microsoft/mana/mana_en.c > index 6b7c027e497ed..2bf17fdcfa2bd 100644 > --- a/drivers/net/ethernet/microsoft/mana/mana_en.c > +++ b/drivers/net/ethernet/microsoft/mana/mana_en.c [ ... ] > @@ -3774,42 +3784,104 @@ static int mana_dealloc_queues(struct net_device > *ndev) > * a txq because it may not timely see apc->port_is_up being cleared > * to false, but it doesn't matter since mana_start_xmit() drops any > * new packets due to apc->port_is_up being false. > - * > - * Drain all the in-flight TX packets. > - * A timeout of 120 seconds for all the queues is used. > - * This will break the while loop when h/w is not responding. > - * This value of 120 has been decided here considering max > - * number of queues. > + * Drain the in-flight TX completions for each queue. TX is already > + * stopped, so pending_sends can no longer grow. For each queue, > + * disable NAPI so it cannot run at the same time as this drain, > + * then process all completions still in the CQ. If some are still > + * missing after that, the hardware is not responding: reset device > + * and stop waiting on the now-unavailable HWC. > */ > > if (apc->tx_qp) { > for (i = 0; i < apc->num_queues; i++) { > + struct napi_struct *napi; > + struct mana_cq *tx_cq; > + int pending; > + > + if (!apc->tx_qp[i]) > + continue; > + > + tx_cq = &apc->tx_qp[i]->tx_cq; > txq = &apc->tx_qp[i]->txq; > - tsleep = 1000; > - while (atomic_read(&txq->pending_sends) > 0 && > - time_before(jiffies, timeout)) { > - usleep_range(tsleep, tsleep + 1000); > - tsleep <<= 1; > - } > - if (atomic_read(&txq->pending_sends)) { > - err = mana_gd_dev_reset(gd->gdma_context); > - if (err) { > - netdev_err(ndev, "device reset failed > %d with %d pkts pending in txq %u\n", > - err, > - atomic_read(&txq->pending_sends), > - txq->gdma_txq_id); > + napi = &tx_cq->napi; > + > + if (txq->napi_initialized) { > + /* Stop the NAPI poll so it cannot run > + * concurrently with the synchronous drain. > + */ > + napi_synchronize(napi); > + napi_disable_locked(napi); > + cancel_work_sync(&tx_cq->dim.work); > + > + /* Reap all TX completions currently in the CQ. > + * For completion which have still not arrived > + * allow time to settle. > + * A zero budget frees skbs via the non-NAPI > + * path, which is required outside softirq. > + */ > + tx_cq->budget = 0; > + for (retry = 0; retry < MANA_TX_DRAIN_RETRIES; > + retry++) { > + do { > + tx_cq->work_done = 0; > + mana_poll_tx_cq(tx_cq); > + } while (tx_cq->work_done); > + > + if (!atomic_read(&txq->pending_sends) || > + trigger_flr) > + break; > + > + msleep(100); > } > - break; > } > + > + pending = atomic_read(&txq->pending_sends); > + > + if (!trigger_flr && pending > 0) { > + trigger_flr = true; > + flr_txq_id = txq->gdma_txq_id; > + flr_pending = pending; > + } > + } [Severity: High] In mana_dealloc_queues(), NAPI is disabled on tx_cq via napi_disable_locked() before entering the MANA_TX_DRAIN_RETRIES loop, so only mana_poll_tx_cq() inside this loop can reap CQEs and decrement txq->pending_sends. On the last iteration (retry == MANA_TX_DRAIN_RETRIES - 1), if txq->pending_sends is still non-zero after polling, the loop calls msleep(100) and then exits without calling mana_poll_tx_cq(tx_cq) again. Because NAPI is disabled, any TX completions that arrive during that final 100 ms sleep remain unpolled in the CQ when pending = atomic_read(&txq->pending_sends) is checked right after the loop. Can this cause mana_dealloc_queues() to miss completions that arrived during the final sleep and trigger an unnecessary mana_gd_dev_reset() and hwc->hwc_dead = true? > + > + if (trigger_flr && !(hwc && hwc->hwc_dead)) { > + /* Completions never arrived: the device is > + * unresponsive. Issue device reset and > + * mark the HWC dead so subsequent HWC commands > + * fail fast instead of each waiting the full > + * hwc_timeout. > + */ > + err = mana_gd_dev_reset(gd->gdma_context); > + if (err) > + netdev_err(ndev, "device reset failed %d with > %d pkts pending in txq %u\n", > + err, > + flr_pending, > + flr_txq_id); > + else if (hwc) > + WRITE_ONCE(hwc->hwc_dead, true); > } > > for (i = 0; i < apc->num_queues; i++) { > + struct napi_struct *napi; > + struct mana_cq *tx_cq; > + > + if (!apc->tx_qp[i]) > + continue; > + > + tx_cq = &apc->tx_qp[i]->tx_cq; > + napi = &tx_cq->napi; > + > txq = &apc->tx_qp[i]->txq; > while ((skb = skb_dequeue(&txq->pending_skbs))) { > mana_unmap_skb(skb, apc); > dev_kfree_skb_any(skb); > } > atomic_set(&txq->pending_sends, 0); > + > + if (txq->napi_initialized) { > + netif_napi_del_locked(napi); > + txq->napi_initialized = false; > + } > } > } > > @@ -3822,7 +3894,8 @@ static int mana_dealloc_queues(struct net_device *ndev) > if (err && mana_en_need_log(apc, err)) > netdev_err(ndev, "Failed to disable vPort: %d\n", err); > > - mana_fence_rqs(apc); > + if (!(hwc && hwc->hwc_dead)) > + mana_fence_rqs(apc); [Severity: Medium] Should the dead-HWC checks in mana_dealloc_queues() also account for hwc->hwc_timeout == 0 (or mana_disable_vport_rx() timing out) and check whether the HWC is already dead inside the TX drain retry loop? Specifically: 1. In mana_serv_reset(), an unresponsive HWC is marked by setting WRITE_ONCE(hwc->hwc_timeout, 0) before calling mana_gd_suspend() -> mana_remove() -> mana_detach() -> mana_dealloc_queues(): drivers/net/ethernet/microsoft/mana/gdma_pci.c:mana_serv_reset() { ... WRITE_ONCE(hwc->hwc_timeout, 0); dev_info(&pdev->dev, "MANA reset cycle start\n"); mana_gd_suspend(pdev, PMSG_SUSPEND); } If no TX packets are pending (so trigger_flr is false and hwc->hwc_dead stays false), or if the HWC dies while TX is idle and mana_disable_vport_rx() fails with -ETIMEDOUT (reducing hwc->hwc_timeout to 1 without setting hwc->hwc_dead), checking only !(hwc && hwc->hwc_dead) still calls mana_fence_rqs(apc), which logs an error and sleeps 100 ms per RX queue while holding rtnl_lock. 2. In the TX drain retry loop above, the early break only checks the local variable trigger_flr: if (!atomic_read(&txq->pending_sends) || trigger_flr) break; On a multi-port device during mana_remove(), after the first port resets the device and sets hwc->hwc_dead = true (or when hwc->hwc_timeout == 0), subsequent calls to mana_dealloc_queues() start with trigger_flr == false and will still sleep through MANA_TX_DRAIN_RETRIES on the first queue with pending sends. -- Sashiko AI review ยท https://sashiko.dev/#/patchset/[email protected]?part=1
