On 08/13/2014 10:03 PM, Walid Nouri wrote:
While looking to find some ideas for approaches to replicating block
devices I have read the paper about the Remus implementation. I think
MC can take a similar approach for local disk.
I agree.
Here are the main facts that I have understood:
Local disk contents is viewed as internal state the primary and
secondary.
In the explanation they describe that for keeping disc semantics of
the primary and to allow the primary to run speculatively all disc
state changes are directly written to the disk. In parrallel and
asynchronously send to the secondary. The secondary keeps the pending
writing requests in two disk buffers. A speculation-disk-buffer and a
write-out-buffer.
After the reception of the next checkpoint the secondary copies the
speculation buffer to the write out buffer, commits the checkpoint and
applies the write out buffer to its local disk.
When the primary fails the secondary must wait until write-out-buffer
has been completely written to disk before before changing the
execution mode to run as primary. In this case (failure of primary)
the secondary discards pending disk writes in its speculation buffer.
This protocol keeps the disc state consistent with the last checkpoint.
Remus uses the XEN specific blktap driver. As far as I know this can’t
be used with QEMU (KVM).
I must see how drive-mirror can be used for this kind of protocol.
That's all correct. Theoretically, we would do exactly the same thing:
drive-mirror on the source would write immediately to disk but follow
the same commit semantics on the destination as Xen.
I have taken a look at COLO.
IMHO there are two points. Custom changes of the TCP-Stack are a no-go
for proprietary operating systems like Windows. It makes COLO
application agnostic but not operating system agnostic. The other
point is that with I/O intensive workloads COLO will tend to behave
like MC. This is my point of view but i didn’t invest much time to
understand everything in detail.
Actually, if I remember correctly, the TCP stack is only modified at the
hypervisor level - they are intercepting and translating TCP sequence
numbers "in-flight" to detect divergence of the source and destination -
which is not a big problem if the implementation is well-done.
My hope in the future was that the two approaches could be used in a
"Hybrid" manner - actually MC has much more of a performance hit for I/O
than COLO does because of its buffering requirements.
On the other hand, MC would perform better in a memory-intensive or
CPU-intensive situation - so maybe QEMU could "switch" between the two
mechanisms at different points in time when the resource bottleneck changes.
- Michael