This series picks up Long Li's v5 MANA Hardware Channel (HWC) work, addresses its review findings, and retains its four-patch feature lineage. Safety changes required by concurrency and repeated HWC reinitialization are folded into the patch which owns each behavior rather than posted as a separate fixes series.
I have taken over follow-up upstreaming from Long. Long remains the author of every patch, his existing sign-off is preserved, and I added mine for the changes and submission custody. Patch 1 prepares HWC ownership for safe reinitialization. It serializes complete shared-memory transactions, tracks possible PF ownership, retains mappings after an unacknowledged destroy, fences the HWC EQ before releasing CQ state, and safely publishes and withdraws the HWC CQ table. Patch 2 gives each slot independent completion state, withdraws caller storage before sender return, handles the response-versus-timeout race, and prevents depth-one slot reuse after a genuine timeout. Patch 3 adds concurrent requests with bounded admission, serialized SQ posting, separate sender/response ownership, stale-response quarantine, coherent timeout updates, private setup publication, and sender draining. Patch 4 rebuilds the depth-one bootstrap queues at validated depths from 2 through 128. It validates both reports, message limits and doorbells, keeps the untouched bootstrap channel for depth one or reports above 128, and restores fresh bootstrap queues after a failed rebuild only when teardown was acknowledged. Generic Ethernet/RDMA CQ callback refcounting and generic service-work versus device-lifecycle serialization are independent pre-existing work and are intentionally excluded. Malformed-RX handling, duplicate-success correlation, and broad device-controlled allocation/timeout policy caps also remain separate follow-ups. This series is independent of the separately posted queue-set v6 series. That series addresses the pre-existing generic Ethernet CQ reclamation gap discussed in the HWC v6 review. The CQ-publication hunks overlap; whichever series lands second will need reconciliation preserving both table-publication and individual-CQ lifetime guarantees. https://lore.kernel.org/all/[email protected]/ Changes in v7 (v6 -> v7): - Rebase onto net-next d8674294aefef02266c4d47ad10131f1bffbe534, following Simon Horman's request after the v6 CI application failure. - Preserve the upstream GDMA core/PCI transport split and CDX support. Initialize the HWC lifetime lock on both transport probe paths and perform PCI recovery's timeout cancellation through a GDMA core helper which protects the published HWC pointer across the module boundary. - Retain the four-patch lineage and the v6 ownership, request, timeout, publication, depth-validation and fallback policies. Rebase request: https://lore.kernel.org/all/[email protected]/ Changes in v6 (v5 -> v6): - Keep the four-patch v5 structure while folding review-required HWC ownership, teardown, timeout, publication and fallback safety into their owning feature patches. - Serialize complete ESTABLISH_HWC and DESTROY_HWC shared-memory transactions. Record possible PF ownership before MMIO submission and clear it only after DESTROY_HWC is acknowledged. - Retain PF-visible queues after an indeterminate destroy. Fence the dedicated HWC EQ and withdraw its CQ from dispatch without freeing the mappings, then retry teardown before assigning fresh queues. - Destroy the HWC EQ before CQ and completion state. Release-publish the CQ table and wait for IRQ-side RCU readers before freeing it, without adding per-completion refcount operations. - Withdraw the caller's response buffer before return and quarantine a timed-out slot until its late response arrives or teardown cancels it. - Return -EBUSY for healthy slot pressure and reserve -ETIMEDOUT for an admitted request which receives no response. - Publish slots under the inflight lock, protect response state with the per-slot lock, and retain sender/response references until both sides finish. - Synchronize hwc_timeout updates and publish the HWC only after queues, admission state, caller contexts and the direct EQ test are ready. - Replace the goto-heavy request and channel-creation flows called out in review with focused admission, submission, wait, build, validation, rebuild and fallback helpers. - Treat reported depth as a maximum: rebuild exactly for 2 through 128, retain untouched bootstrap queues for 1 or above 128, and require the rebuilt report to repeat the selected depth. - Enforce directional request/response maxima against both the report and allocated buffers. Rebuild only for exact 4096-byte reports while permitting bootstrap operation for other nonzero maxima. - Validate BAR page geometry and every stored doorbell. Invalidate the old HWC doorbell before re-establishment and suppress MMIO while the invalid sentinel is present, without adding full validation to each data-path doorbell ring. - Remove the proposed capability bit because driver-version advertisement occurs after HWC establishment and cannot negotiate creation-time behavior. - Preserve current net-next behavior which always uses the hardware-reported dest_vrq_id and dest_vrcq_id. Changes in v5 (v4 -> v5): - Shortened comments and documented the existing error paths in patches 1 and 2. - Rejected apparent success from a response accepted before request submission. - Guarded CQ unpublication when establishment failed before cq_table allocation. - Required successful retry teardown before creating fresh bootstrap queues after an initial destroy failure. Changes in v4 (return to net-next after the v3 split): - Reworked the feature series into four standalone patches for handover, per-slot completion state, concurrency and dynamic depth. - Added bounded admission, guarded sender accounting, timeout quarantine, queue-depth bounds, message-buffer sizing and rebuild validation. - Reset routing IDs for each establish and rejected a missing doorbell. Changes in v3 (v2 -> net v3, historical fixes-only posting): - Split the combined series after maintainer feedback, posting fixes to net and deferring concurrency and dynamic depth. - Added response-buffer withdrawal and stale-response handling. - Removed pcie_flr() teardown fallback and retained resources after an unacknowledged destroy. Changes in v2 (v1 -> v2): - Bounds-checked the SGE pointer derived from inline OOB size. - Protected active-sender accounting and drained senders during teardown and failure exits. - Rebased on net-next e354f7d60f14. v1: - Combined CQ-table lifetime, queue sizing, RX validation, teardown, concurrent-request and dynamic-depth work in seven patches. Validation: - Exact application and strict checkpatch for all four patches. - Baseline and every intermediate commit built with GCC/Clang W=1, including GDMA core, PCI, CDX COMPILE_TEST and RDMA target objects. - Full baseline/release kernel and module builds on the new base. - Sparse: no new diagnostics relative to baseline. Smatch remains limited by identical baseline frontend and array-bound diagnostics; it is not counted as a clean pass. - Fresh baseline/release runs on two Azure MANA VMs: concurrent PD/MR and native RNIC PD/CQ/MR resource controls, channel/MTU changes, three PCI remove/rescan cycles per VM, and three samples each of forward/reverse TCP and 64-byte UDP. No new kernel warning or functional failure in these runs; safe kernels/configurations restored. - These runs do not establish negotiated multi-request depth, connected RDMA data-plane operation, CDX hardware support or fresh firmware-fault, fallback, sanitizer/lockdep coverage. Prior v6 fault evidence is not presented as fresh v7 execution. v6: https://lore.kernel.org/all/[email protected]/ v5: https://lore.kernel.org/all/[email protected]/ v4: https://lore.kernel.org/all/[email protected]/ v3: https://lore.kernel.org/all/[email protected]/ v2: https://lore.kernel.org/all/[email protected]/ v1: https://lore.kernel.org/all/[email protected]/ Long Li (4): net: mana: prepare HWC ownership for safe reinitialization net: mana: give each HWC message slot its own completion state net: mana: support concurrent HWC requests net: mana: add dynamic HWC queue depth with reinit path .../net/ethernet/microsoft/mana/gdma_cdx.c | 1 + .../net/ethernet/microsoft/mana/gdma_main.c | 140 ++- .../net/ethernet/microsoft/mana/gdma_pci.c | 11 +- .../net/ethernet/microsoft/mana/hw_channel.c | 928 +++++++++++++++--- .../net/ethernet/microsoft/mana/shm_channel.c | 58 +- include/net/mana/gdma.h | 16 + include/net/mana/hw_channel.h | 56 +- include/net/mana/shm_channel.h | 10 +- 8 files changed, 1019 insertions(+), 201 deletions(-) base-commit: d8674294aefef02266c4d47ad10131f1bffbe534 -- 2.43.0
