On 2026-09-23 17:20, Mukul Joshi wrote:
The export cleanup worker checks amdgpu_ualink_check_conn_ready() once
when building npa_release_bitmap, but not again in the send loop. Cleanup
workers run concurrently on the unbounded NPA WQ (one per exported
handle), so when a peer's egress dies many workers have already seeded
npa_release_bitmap from a connection that was still ESTABLISHED and all
issue an LSDMA copy to the dead peer, flooding the log with repeated
NPA-REVOKE send failures.
Re-check the connection immediately before each send, as the tlb
shootdown, force_retry_rpcs and unmap_all_npa_addr loops already do, so
once the first worker marks a peer NOT_READY the rest skip it. With the
re-check in place, seed npa_release_bitmap with a single bitmap_copy() of
importers_bitmap instead of a separate build loop; peers that fail the
re-check or the send are dropped from the bitmap in the send loop.
Signed-off-by: Mukul Joshi <[email protected]>
---
v1->v2:
- We were using 2 for loops, once for building the npa_release_bitmap,
and another for sending the NPA-REVOKEs.
- Replace it by using a single loop instead.
drivers/gpu/drm/amd/amdgpu/amdgpu_ualink.c | 31 +++++++++++++---------
1 file changed, 19 insertions(+), 12 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ualink.c
b/drivers/gpu/drm/amd/amdgpu/amdgpu_ualink.c
index 39b7cc8bc1c5..f6ad64be4349 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ualink.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ualink.c
@@ -2936,26 +2936,34 @@ static void amdgpu_ualink_exp_cleanup_worker(struct
work_struct *work)
amdgpu_bo_unref(&bo);
exp_xa_node->bo = NULL;
- /* Build the full npa_release_bitmap before sending any NPA-REVOKE. */
+ /* Seed npa_release_bitmap with all importers before any NPA-REVOKE, so
+ * the release IRQ only sees it empty once every responder has replied.
+ * Dead peers are dropped from it in the send loop below.
+ */
+ bitmap_copy(exp_xa_node->npa_release_bitmap,
+ exp_xa_node->importers_bitmap, AMDGPU_UALINK_ACCEL_MAX);
+
+ /* Send NPA-REVOKE to all importers which have imported this memory.
+ * On send failure clear the bit (no response will arrive) and mark the
+ * connection NOT_READY so other cleanups skip this unreachable peer.
+ */
for_each_set_bit(remote_acc_id, exp_xa_node->importers_bitmap,
AMDGPU_UALINK_ACCEL_MAX) {
imp_entry = &exp_xa_node->importer_entries[remote_acc_id];
+
+ /* Re-check the connection right before the send: concurrent
+ * cleanup workers all seed the bitmap while the peer is still
+ * ESTABLISHED, so once its egress dies and the first worker
+ * marks it NOT_READY, this skips the rest instead of flooding
+ * the log with failed LSDMA sends to the same dead peer.
+ */
if (!amdgpu_ualink_check_conn_ready(adev, remote_acc_id,
imp_entry->generation_count)) {
clear_bit(remote_acc_id,
- exp_xa_node->importers_bitmap);
+ exp_xa_node->npa_release_bitmap);
It look like you never clear bits in importers_bitmap now. So maybe you
could get rid of orig_importers_bitmap now as well. But that could be a
separate cleanup patch if this one is urgent.
Either way, this patch is
Reviewed-by: Felix Kuehling <[email protected]>
continue;
}
- set_bit(remote_acc_id, exp_xa_node->npa_release_bitmap);
- }
-
- /* Send NPA-REVOKE to all importers which have imported this memory.
- * On send failure clear the bit (no response will arrive) and mark the
- * connection NOT_READY so other cleanups skip this unreachable peer.
- */
- for_each_set_bit(remote_acc_id, exp_xa_node->importers_bitmap,
- AMDGPU_UALINK_ACCEL_MAX) {
dev_dbg(adev->dev,
"EXP-CLEANUP: Sending NPA-REVOKE to remote:%u\n",
remote_acc_id);
@@ -2966,7 +2974,6 @@ static void amdgpu_ualink_exp_cleanup_worker(struct
work_struct *work)
remote_acc_id);
clear_bit(remote_acc_id,
exp_xa_node->npa_release_bitmap);
- imp_entry = &exp_xa_node->importer_entries[remote_acc_id];
amdgpu_ualink_handle_connection_reset(adev,
remote_acc_id,
AMDGPU_UALINK_CONN_NOT_READY,
imp_entry->generation_count);
