The branch main has been updated by jhb:
URL:
https://cgit.FreeBSD.org/src/commit/?id=6751f65e6af15348abdc6106cf54c8335d45e49b
commit 6751f65e6af15348abdc6106cf54c8335d45e49b
Author: John Baldwin <j...@freebsd.org>
AuthorDate: 2024-10-31 15:03:41 +0000
Commit: John Baldwin <j...@freebsd.org>
CommitDate: 2025-01-13 20:04:29 +0000
nvmf: Defer the post-sync shutdown handler to SHUTDOWN_PRI_LAST
nda(4) has its own shutdown handler that runs at SHUTDOWN_PRI_DEFAULT
that calls ndaflush() that could run after the nvmf handler. Instead,
give a the flush a chance to run before the graceful shutdown of the
controller.
While here, be a bit more defensive in the post-sync case and shutdown
the consumers (sim and /dev/nvmeXnY devices) before destroying the
queue pairs so that if any requests are submitted after the post-sync
handler they fail gracefully instead of trying to use a destroyed
queue pair.
Reported by: Sony Arpita Das <sonyarpi...@chelsio.com>
Sponsored by: Chelsio Communications
---
sys/dev/nvmf/host/nvmf.c | 14 +++++++++++++-
1 file changed, 13 insertions(+), 1 deletion(-)
diff --git a/sys/dev/nvmf/host/nvmf.c b/sys/dev/nvmf/host/nvmf.c
index 09d5cecdfad6..77d3081243f6 100644
--- a/sys/dev/nvmf/host/nvmf.c
+++ b/sys/dev/nvmf/host/nvmf.c
@@ -554,7 +554,7 @@ nvmf_attach(device_t dev)
sc->shutdown_pre_sync_eh = EVENTHANDLER_REGISTER(shutdown_pre_sync,
nvmf_shutdown_pre_sync, sc, SHUTDOWN_PRI_FIRST);
sc->shutdown_post_sync_eh = EVENTHANDLER_REGISTER(shutdown_post_sync,
- nvmf_shutdown_post_sync, sc, SHUTDOWN_PRI_FIRST);
+ nvmf_shutdown_post_sync, sc, SHUTDOWN_PRI_LAST);
return (0);
out:
@@ -776,6 +776,18 @@ nvmf_shutdown_post_sync(void *arg, int howto)
callout_drain(&sc->ka_rx_timer);
nvmf_shutdown_controller(sc);
+
+ /*
+ * Quiesce consumers so that any commands submitted after this
+ * fail with an error. Notably, nda(4) calls nda_flush() from
+ * a post_sync handler that might be ordered after this one.
+ */
+ for (u_int i = 0; i < sc->cdata->nn; i++) {
+ if (sc->ns[i] != NULL)
+ nvmf_shutdown_ns(sc->ns[i]);
+ }
+ nvmf_shutdown_sim(sc);
+
for (u_int i = 0; i < sc->num_io_queues; i++) {
nvmf_destroy_qp(sc->io[i]);
}
