On Fri, Oct 16, 2026 at 05:42:42PM -0400, Gregory Price (Meta) wrote: > Kernels with the dax_kmem daxX.Y/state attribute [1] can unplug or > plug-and-online a whole device atomically. Use it in daxctl > online-memory and offline-memory so other agents (udev, auto-online) > can't race daxctl block by block, notably the offline-then-unbind race > on teardown. > > Kernels without the attribute keep the per-block path. > > Behavior change: offline-memory now always unplugs the device (state > 'unplugged'), removing its memory blocks, even if they were all > offline already. Previously that was a no-op. > > 'daxctl list' (and 'ndctl/cxl list -X') gains 'hotplug_state', > reporting daxX.Y/state. The kernel only updates that when written, so > it goes stale if blocks are toggled individually. > > If there is a mismatch between 'dax/state' and the blocks, then > 'hotplug_state' reports 'inconsistent' (online state / zones disagree). > That is intentional: it flags agents fighting over the device. > It only appears when the attribute exists. > > Compatibility notes: > - Where the unplug fails (e.g. no memory hotremove support), > online-memory falls back to the per-block path and offline-memory > of an already offline device remains a no-op. > - Older daxctl/ndctl/cxl binaries don't expect a device with no memory > blocks. Basically multiple daxctl versions on the same box can cause > older versions to fail when in the unplugged state. Not a huge issue. > - If re-adding fails after an unplug (e.g. region can't be reserved) > the device is left unplugged. > - For multi-range devices the kernel succeeds if any range is added. > daxctl can't tell that a range is missing. > > [1] https://lore.kernel.org/all/[email protected]/
Hi Greg, Next version, please add the ndctl label, ie [ndctl PATCH v2 0/6] (https://github.com/pmem/ndctl?tab=contributing-ov-file) I see this goes with the kernel functionality that arrived in kernel 7.3. That took me a bit to realized because the linked kernel series shows crickets at v7. I'd like to get this into the pending ndctl release, so let's see if we can close on a few things. I tested against 7.3, regression and new tests and all is good there. I tested against 7.2 because daxctl command functionality itself needs to be backward compatible, and it is! Some per patch feedback and tags follow. -- Alison > > Gregory Price (6): > libdaxctl: add dax kmem atomic hotplug state accessors > daxctl: online whole-device memory atomically when available > daxctl: offline whole-device memory via atomic unplug when available > daxctl: report whole-device hotplug state in 'daxctl list' > Documentation: describe atomic whole-device hotplug in daxctl man > pages > test/daxctl-devices.sh: cycle offline/online to cover unplug and > replug > > Documentation/daxctl/daxctl-list.txt | 5 + > .../daxctl/daxctl-offline-memory.txt | 5 + > Documentation/daxctl/daxctl-online-memory.txt | 5 + > .../daxctl/daxctl-reconfigure-device.txt | 5 +- > daxctl/device.c | 156 +++++++++++++++++- > daxctl/json.c | 53 +++++- > daxctl/lib/libdaxctl-private.h | 9 + > daxctl/lib/libdaxctl.c | 71 +++++++- > daxctl/lib/libdaxctl.sym | 6 + > daxctl/libdaxctl.h | 18 ++ > test/daxctl-devices.sh | 53 ++++++ > 11 files changed, 379 insertions(+), 7 deletions(-) > > -- > 2.53.0-Meta > >
