During an array controller failover on an iSCSI host, multipathd briefly put a path to one LUN into the map of another LUN, and I/O meant for the first LUN landed on the second. The host ran RHEL 9 device-mapper-multipath 0.8.7, but the code involved is essentially unchanged in master. See the v1 cover letter for the full sequence.
Changes in v2, after Ben's review: - ev_add_path() no longer double-checks the WWID on every add. Ben was concerned about adding a check to perfectly normal adds, since a SCSI device can report several identifiers in VPD 0x83 and multipath's SG_IO code has to pick the same one udev does. The need_do_map argument of uev_add_path() and ev_add_path() is now enum path_add_mode, with a new PATH_ADD_REINIT value. It behaves like PATH_ADD_MAP, and is passed only when a path is re-added from uev_update_path(): after udev reported a changed WWID, or after udev data was missing at first. Only then is the WWID checked against the device. Ordinary adds never run the SG_IO check. - The uev_update_path() check is unchanged. It only runs when the WWID from udev differs from the cached one, and falls back to the old behavior if the device doesn't confirm the cached WWID. About how likely this is: we hit it because a test tool was opening and closing the sd devices read-write many times a second, and with udev's "watch" rule every close generates a change uevent that runs scsi_id against the device. That is far more udev traffic than a normal host generates, so we expect the window to be rare in practice. We don't know why the bad WWID was reported; the array may have answered the INQUIRY wrongly during the takeover, or the bad value may have arisen elsewhere in the udev path. What the patch does is stop multipathd from acting on a changed WWID without confirming it, since the result of acting on a wrong one is data going to the wrong LUN. Open points, as in v1: - If the SG_IO read fails, behavior is unchanged. Refusing the add in that case would be stricter; opinions welcome. - "multipathd reconfigure" builds maps through configure(), not ev_add_path(), so it isn't covered. Brian Bunker (1): multipathd: verify udev WWIDs with the device before moving paths multipathd/cli_handlers.c | 2 +- multipathd/main.c | 79 +++++++++++++++++++++++++++++++-------- multipathd/main.h | 12 +++++- 3 files changed, 76 insertions(+), 17 deletions(-) -- 2.56.0
