This patchset implements iSCSI multipath support in sheepdog. Below is a brief description of the problem and solution (mainly quoting from 2nd patch):
The main problem of enabling iSCSI multipath for sheepdog's VDI is keeping coherence of tgtd's internal inode data structure. Assume tgt A and B have logical units which point same sheepdog VDI. After initialization of the logical units, A and B have copies of inode object in their address space (let's X denote this object). When an initiator sends write request to A and the request causes COW, A creates a new object and updates the inode object X (let's X' denote the updated one). After the update by A, A can die with hardware faults. In such a case, the initiator detects a connection error and switch its target to B if iSCSI multipath is configured correctly. But the problem can arise at this point. Because B doesn't know the update caused by A and its internal inode object is still X, not X'. B creates requests based on X, so it can read obsolete data which is already hidden by X' or update X' in wrong manner. For solving the above problem, this patchset implements a mechanism for keeping inode objects consistent in clients' address spaces. The above problem is very similar to cache coherence problem of shared memory multicore processors, so the mechanism mocks a simple MSI protocol. When VDI is locked with shared mode for iSCSI multipath, sheep's internal data structure for representing locking has 3 states (Modified, Shared, and Invalidated) per participating node (in the above example, A and B). The above states are updated via gateway read and write of inode objects: read makes states of every node shared, write makes writer's state modified and others invalidated. With the above protocol, sheepdog can prevent requests from tgt based on obsolete inode object. Corresponding changes for QEMU and tgt can be found: - QEMU: https://github.com/sheepdog/qemu/tree/iscsi-multipath -- change for QEMU is just for adopting protocol update - tgt: https://github.com/sheepdog/tgt/tree/iscsi-multipath Hitoshi Mitake (4): sheepdog proto: a new field type in vdi request for locking sheep, dog: introduce a new state for expressing coherence of inode objects in clients sheep: implement coherence protocol of inode objects in address spaces of clients sheep: handle node leave event for updating inode object shared state dog/vdi.c | 32 ++- include/internal_proto.h | 28 ++- include/sheepdog_proto.h | 10 + sheep/gateway.c | 24 ++- sheep/group.c | 2 + sheep/ops.c | 33 ++- sheep/sheep_priv.h | 14 +- sheep/vdi.c | 518 ++++++++++++++++++++++++++++++++++++++++++----- 8 files changed, 588 insertions(+), 73 deletions(-) -- 1.8.3.2 -- sheepdog mailing list sheepdog@lists.wpkg.org http://lists.wpkg.org/mailman/listinfo/sheepdog
