Kernels with the dax_kmem daxX.Y/state attribute [1] can unplug or plug-and-online a whole device atomically. Use it in daxctl online-memory and offline-memory so other agents (udev, auto-online) can't race daxctl block by block, notably the offline-then-unbind race on teardown.
Kernels without the attribute keep the per-block path. Behavior change: offline-memory now always unplugs the device (state 'unplugged'), removing its memory blocks, even if they were all offline already. Previously that was a no-op. 'daxctl list' (and 'ndctl/cxl list -X') gains 'hotplug_state', reporting daxX.Y/state. The kernel only updates that when written, so it goes stale if blocks are toggled individually. If there is a mismatch between 'dax/state' and the blocks, then 'hotplug_state' reports 'inconsistent' (online state / zones disagree). That is intentional: it flags agents fighting over the device. It only appears when the attribute exists. Compatibility notes: - Where the unplug fails (e.g. no memory hotremove support), online-memory falls back to the per-block path and offline-memory of an already offline device remains a no-op. - Older daxctl/ndctl/cxl binaries don't expect a device with no memory blocks. Basically multiple daxctl versions on the same box can cause older versions to fail when in the unplugged state. Not a huge issue. - If re-adding fails after an unplug (e.g. region can't be reserved) the device is left unplugged. - For multi-range devices the kernel succeeds if any range is added. daxctl can't tell that a range is missing. [1] https://lore.kernel.org/all/[email protected]/ Gregory Price (6): libdaxctl: add dax kmem atomic hotplug state accessors daxctl: online whole-device memory atomically when available daxctl: offline whole-device memory via atomic unplug when available daxctl: report whole-device hotplug state in 'daxctl list' Documentation: describe atomic whole-device hotplug in daxctl man pages test/daxctl-devices.sh: cycle offline/online to cover unplug and replug Documentation/daxctl/daxctl-list.txt | 5 + .../daxctl/daxctl-offline-memory.txt | 5 + Documentation/daxctl/daxctl-online-memory.txt | 5 + .../daxctl/daxctl-reconfigure-device.txt | 5 +- daxctl/device.c | 156 +++++++++++++++++- daxctl/json.c | 53 +++++- daxctl/lib/libdaxctl-private.h | 9 + daxctl/lib/libdaxctl.c | 71 +++++++- daxctl/lib/libdaxctl.sym | 6 + daxctl/libdaxctl.h | 18 ++ test/daxctl-devices.sh | 53 ++++++ 11 files changed, 379 insertions(+), 7 deletions(-) -- 2.53.0-Meta
