During an array controller failover on an iSCSI host, multipathd briefly put a path to one LUN into the map of another LUN, and I/O meant for the first LUN landed on the second. The host ran RHEL 9 device-mapper-multipath 0.8.7, but the code involved is essentially unchanged in master.
What happened: 1. A change uevent for sdq (verify3) carried verify1's WWID. The next uevent 4.5 s later carried the correct one. We don't know whether the array answered the INQUIRY (VPD 0x83, run by scsi_id) wrongly during the takeover or the bad value arose elsewhere in the udev path; we have no packet capture. 2. uev_update_path() acted on the uevent WWID without asking the device and removed sdq from verify3's map. That reload blocked for 4.4 s behind I/O stuck in the failover. 3. uev_update_path() then called uev_add_path() with the same, now stale, uevent. ev_add_path() put sdq into verify1's map, where it stayed for about 170 ms. Test tool records show a write meant for verify1 landing on verify3, and a verify1 read returning verify3's data. recheck_wwid was off, but would not have helped: it is only consulted when the checker sees a failed path come back and on "multipathd add path", never for uevents. The patch reads VPD 0x83 over SG_IO in uev_update_path() and again in ev_add_path(), for paths where can_recheck_wwid() is true. If the device still reports the cached WWID, the udev value is ignored. If the device doesn't confirm the WWID a path is being added with, the path is left out of the map and udev is retriggered. Open points: - If the SG_IO read fails, the add proceeds as before. Refusing the add would be stricter; opinions welcome. - The checks depend on can_recheck_wwid(), not on the recheck_wwid option, because a mismatch at add time means data corruption and recheck_wwid is off by default. - "multipathd reconfigure" builds maps through configure(), not ev_add_path(), so it isn't covered. Brian Bunker (1): multipathd: verify udev WWIDs with the device before moving paths multipathd/main.c | 57 ++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 52 insertions(+), 5 deletions(-) -- 2.56.0
