Problem When we read from an indirect vdev that points to a mirror vdev, we only consider one copy of the data. This can lead to reduced effective redundancy, because we might read a bad copy of the data from one side of the mirror, and not retry the other, good side of the mirror.
Note that the problem is not with the removal process, but rather after the removal has completed (having copied correct data to both sides of the mirror), if one side of the mirror is silently damaged, we encounter the problem when reading the relocated data via the indirect vdev. Also note that the problem doesn't occur when ZFS knows that one side of the mirror is bad, e.g. when a disk entirely fails or is offlined. The problem manifests itself in several ways: * reads (from indirect vdevs that point to mirrors) may return a checksum error even though the good data exists on one side of the mirror * scrub doesn't repair all data on the mirror (if some of it is pointed to via an indirect vdev) Solution As an expedient fix, we will do the following: * When reading a non-split block, the indirect vdev will pass the whole BP down to the mirror code. This lets the mirror code deal with verifying the checksum and trying other copies, etc. So for non-split blocks (typically >90% of all relocated blocks), the problem is completely solved. * When reading split blocks, read the left side of all mirrors, and if the checksum of that fails, read the right side of all mirrors (and the 3rd side for 3-way mirrors, etc). This ensures that if there is a good copy of the data, we will return it. However, it doesn't issue repair i/os, and it doesn't fix scrub for indirect split blocks. A full fix will be the subject of a later PR. You can view, comment on, or merge this pull request online at: https://github.com/openzfs/openzfs/pull/561 -- Commit Summary -- * DLPX-57225 Reads from indirect mirrors only find one copy -- File Changes -- M usr/src/uts/common/fs/zfs/sys/zio.h (1) M usr/src/uts/common/fs/zfs/vdev_indirect.c (122) M usr/src/uts/common/fs/zfs/vdev_mirror.c (23) M usr/src/uts/common/fs/zfs/zio.c (1) -- Patch Links -- https://github.com/openzfs/openzfs/pull/561.patch https://github.com/openzfs/openzfs/pull/561.diff -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/openzfs/openzfs/pull/561 ------------------------------------------ openzfs-developer Archives: https://openzfs.topicbox.com/groups/developer/discussions/T05e845542b9791ed-M941b9cb520fe9affee6d3a43 Powered by Topicbox: https://topicbox.com
