On Mon, Aug 18, 2014 at 01:59:28PM +0800, Liu Yuan wrote: > On Fri, Aug 15, 2014 at 03:59:04PM +0200, Benoît Canet wrote: > > The Friday 15 Aug 2014 à 13:05:17 (+0800), Liu Yuan wrote : > > > This patch adds single read pattern to quorum driver and quorum vote is > > > default > > > pattern. > > > > > > For now we do a quorum vote on all the reads, it is designed for > > > unreliable > > > underlying storage such as non-redundant NFS to make sure data integrity > > > at the > > > cost of the read performance. > > > > > > For some use cases as following: > > > > > > VM > > > -------------- > > > | | > > > v v > > > A B > > > > > > Both A and B has hardware raid storage to justify the data integrity on > > > its own. > > > So it would help performance if we do a single read instead of on all the > > > nodes. > > > Further, if we run VM on either of the storage node, we can make a local > > > read > > > request for better performance. > > > > > > This patch generalize the above 2 nodes case in the N nodes. That is, > > > > > > vm -> write to all the N nodes, read just one of them. If single read > > > fails, we > > > try to read next node in FIFO order specified by the startup command. > > > > > > The 2 nodes case is very similar to DRBD[1] though lack of auto-sync > > > functionality in the single device/node failure for now. But compared > > > with DRBD > > > we still have some advantages over it: > > > > > > - Suppose we have 20 VMs running on one(assume A) of 2 nodes' DRBD backed > > > storage. And if A crashes, we need to restart all the VMs on node B. But > > > for > > > practice case, we can't because B might not have enough resources to > > > setup 20 VMs > > > at once. So if we run our 20 VMs with quorum driver, and scatter the > > > replicated > > > images over the data center, we can very likely restart 20 VMs without any > > > resource problem. > > > > > > After all, I think we can build a more powerful replicated image > > > functionality > > > on quorum and block jobs(block mirror) to meet various High Availibility > > > needs. > > > > > > E.g, Enable single read pattern on 2 children, > > > > > > -drive driver=quorum,children.0.file.filename=0.qcow2,\ > > > children.1.file.filename=1.qcow2,read-pattern=fifo,vote-threshold=1 > > > > > > [1] http://en.wikipedia.org/wiki/Distributed_Replicated_Block_Device > > > > > > Cc: Benoit Canet <ben...@irqsave.net> > > > Cc: Eric Blake <ebl...@redhat.com> > > > Cc: Kevin Wolf <kw...@redhat.com> > > > Cc: Stefan Hajnoczi <stefa...@redhat.com> > > > Signed-off-by: Liu Yuan <namei.u...@gmail.com> > > > --- > > > block/quorum.c | 176 > > > ++++++++++++++++++++++++++++++++++++++++++--------------- > > > 1 file changed, 129 insertions(+), 47 deletions(-) > > > > > > diff --git a/block/quorum.c b/block/quorum.c > > > index d5ee9c0..1235d7c 100644 > > > --- a/block/quorum.c > > > +++ b/block/quorum.c > > > @@ -24,6 +24,7 @@ > > > #define QUORUM_OPT_VOTE_THRESHOLD "vote-threshold" > > > #define QUORUM_OPT_BLKVERIFY "blkverify" > > > #define QUORUM_OPT_REWRITE "rewrite-corrupted" > > > +#define QUORUM_OPT_READ_PATTERN "read-pattern" > > > > > > /* This union holds a vote hash value */ > > > typedef union QuorumVoteValue { > > > @@ -74,6 +75,8 @@ typedef struct BDRVQuorumState { > > > bool rewrite_corrupted;/* true if the driver must rewrite-on-read > > > corrupted > > > * block if Quorum is reached. > > > */ > > > + > > > + QuorumReadPattern read_pattern; > > > } BDRVQuorumState; > > > > > > typedef struct QuorumAIOCB QuorumAIOCB; > > > @@ -117,6 +120,7 @@ struct QuorumAIOCB { > > > > > > bool is_read; > > > int vote_ret; > > > + int child_iter; /* which child to read in fifo pattern */ > > > > I don't understand what "fifo pattern" could mean for a bunch of disk > > as they are not forming a queue. > > Naming isn't 100% accurate but as in Eric's comment (see below), both FIFO and > Round-Robin can be used for two different patterns. > > > Maybe round-robin is more suitable but your code does not implement > > round-robin since it will alway start from the first disk. > > > > Your code is scanning the disks set it's a scan pattern. > > > > That said is it a problem that the first disk will be accessed more often > > than the other ? > > As my commit log documented, the purpose of the read pattern I added is to > speed up read against quorum original read pattern. And the use case is clear > (I hope so) and you can take DRBD as a good example for why we need it. Of > course we are far away from DRBD, which need a recovery logic after all kinds > of > failures. My patch set can be taken as a prelimitary step to implement a DRBD > like service driver. > > Eric previously commented on two read patterns that might be useful: > > "Should we offer multiple modes in addition to 'quorum'? For example, I > could see a difference between 'fifo' (favor read from the first quorum > member always, unless it fails, good when the first member is local and > other member is remote) and 'round-robin' (evenly distribute reads; each > read goes to the next available quorum member, good when all members are > equally distant)." > > > You will have to care to insert disks in different order on each QEMU to > > spread the load. > > This is another use case that my patch set didn't try to solve. > > > Shouldn't the code try to spread the load by circling on the disk like a > > real round robin pattern ? > > Probably not on my patch set, but we can add a yet another round robin pattern > if anyone is intrested.
Benoit, ping...