Re: [Qemu-devel] Microcheckpointing: Memory-VCPU / Disk State consistency

2014-09-25 Thread Walid Nouri
Am 24.09.2014 10:47, schrieb Stefan Hajnoczi: I think the assumption with drive-mirror is that you throw away the destination image if something fails. That's the exact opposite of MC where we want to fail over to the destination :). This was not obivous for me... Here is one example of a

Re: [Qemu-devel] Microcheckpointing: Memory-VCPU / Disk State consistency

2014-09-23 Thread Walid Nouri
Am 18.09.2014 15:56, schrieb Stefan Hajnoczi: There is the issue of request ordering (using write cache flushes). The secondary probably needs to perform requests in the same order and interleave cache flushes in the same way as the primary. Otherwise a power failure on the secondary could

Re: [Qemu-devel] Microcheckpointing: Memory-VCPU / Disk State consistency

2014-09-18 Thread Stefan Hajnoczi
On Wed, Sep 17, 2014 at 10:53:32PM +0200, Walid Nouri wrote: Writing data safely to disk can take milliseconds. Not sure how that figures into your commit step, but I guess commit needs to be fast. We have no time to waste ;) but the disk semantic at the primary should be kept as expected

Re: [Qemu-devel] Microcheckpointing: Memory-VCPU / Disk State consistency

2014-09-17 Thread Walid Nouri
Thank you for your Time and the detailed answer! I have needed some time to work through your answer ;-) What MC needs is a block device agnostic, controlled and asynchronous approach for replicating the contents of block devices and its state changes to the secondary VM while the primary VM is

Re: [Qemu-devel] Microcheckpointing: Memory-VCPU / Disk State consistency

2014-09-12 Thread Stefan Hajnoczi
On Thu, Sep 11, 2014 at 06:44:08PM +0100, Dr. David Alan Gilbert wrote: (I've cc'd in Fam, Stefan, and Kevin for Block stuff, and Yang and Eddie for Colo) * Walid Nouri (walid.no...@gmail.com) wrote: Hello Michael, Hello Paolo i have ???studied??? the available

Re: [Qemu-devel] Microcheckpointing: Memory-VCPU / Disk State consistency

2014-09-11 Thread Paolo Bonzini
Il 10/09/2014 17:43, Walid Nouri ha scritto: The drive-mirror (and live migration) does not rely on shared storage and allow live block device copy and incremental syncing. I think your analysis is right. However, just for completeness I'll note that quorum doesn't need shared storage. To do

Re: [Qemu-devel] Microcheckpointing: Memory-VCPU / Disk State consistency

2014-09-11 Thread Dr. David Alan Gilbert
(I've cc'd in Fam, Stefan, and Kevin for Block stuff, and Yang and Eddie for Colo) * Walid Nouri (walid.no...@gmail.com) wrote: Hello Michael, Hello Paolo i have ???studied??? the available documentation/Information and tried to get an idea of the QEMU live block operation

Re: [Qemu-devel] Microcheckpointing: Memory-VCPU / Disk State consistency

2014-09-11 Thread Walid Nouri
Am 11.09.2014 19:44, schrieb Dr. David Alan Gilbert: For keeping the complete system state consistent on the secondary system there must be a possibility for MC to commit/discard block device state changes. In normal operation the mirrored block device state changes (block buffer) are committed

Re: [Qemu-devel] Microcheckpointing: Memory-VCPU / Disk State consistency

2014-09-11 Thread Hongyang Yang
在 09/12/2014 01:44 AM, Dr. David Alan Gilbert 写道: (I've cc'd in Fam, Stefan, and Kevin for Block stuff, and Yang and Eddie for Colo) * Walid Nouri (walid.no...@gmail.com) wrote: Hello Michael, Hello Paolo i have ???studied??? the available documentation/Information and tried

Re: [Qemu-devel] Microcheckpointing: Memory-VCPU / Disk State consistency

2014-09-11 Thread Hongyang Yang
在 09/11/2014 09:50 AM, Michael R. Hines 写道: On 09/10/2014 11:43 PM, Walid Nouri wrote: Hello Michael, Hello Paolo i have „studied“ the available documentation/Information and tried to get an idea of the QEMU live block operation possibilities. I think the MC protocol doesn’t need synchronous

Re: [Qemu-devel] Microcheckpointing: Memory-VCPU / Disk State consistency

2014-09-10 Thread Walid Nouri
Hello Michael, Hello Paolo i have „studied“ the available documentation/Information and tried to get an idea of the QEMU live block operation possibilities. I think the MC protocol doesn’t need synchronous block device replication because primary and secondary VM are not synchronous. The

Re: [Qemu-devel] Microcheckpointing: Memory-VCPU / Disk State consistency

2014-09-10 Thread Michael R. Hines
On 09/10/2014 11:43 PM, Walid Nouri wrote: Hello Michael, Hello Paolo i have „studied“ the available documentation/Information and tried to get an idea of the QEMU live block operation possibilities. I think the MC protocol doesn’t need synchronous block device replication because primary

Re: [Qemu-devel] Microcheckpointing: Memory-VCPU / Disk State consistency

2014-08-19 Thread Walid Nouri
Hi, I have tried to find more information on how to use drive-mirror besides what is available on the wiki. This was not very satisfactory... This may sound naive but are there some code examples in c or any other language, documentation of any kind, blog entries (developer), presentation

Re: [Qemu-devel] Microcheckpointing: Memory-VCPU / Disk State consistency

2014-08-19 Thread Walid Nouri
Hi Paolo, thanks for your hint. I missed your mail from last sunday. I will take a look on that! Walid Am 17.08.2014 um 11:52 schrieb Paolo Bonzini pbonz...@redhat.com: Il 11/08/2014 22:15, Michael R. Hines ha scritto: Excellent question: QEMU does have a feature called drive-mirror in

Re: [Qemu-devel] Microcheckpointing: Memory-VCPU / Disk State consistency

2014-08-17 Thread Paolo Bonzini
Il 11/08/2014 22:15, Michael R. Hines ha scritto: Excellent question: QEMU does have a feature called drive-mirror in block/mirror.c that was introduced a couple of years ago. I'm not sure what the adoption rate of the feature is, but I would start with that one. block/mirror.c is

Re: [Qemu-devel] Microcheckpointing: Memory-VCPU / Disk State consistency

2014-08-14 Thread Dr. David Alan Gilbert
cc'ing in a couple of the COLOers. * Michael R. Hines (mrhi...@linux.vnet.ibm.com) wrote: On 08/13/2014 10:03 PM, Walid Nouri wrote: While looking to find some ideas for approaches to replicating block devices I have read the paper about the Remus implementation. I think MC can take a

Re: [Qemu-devel] Microcheckpointing: Memory-VCPU / Disk State consistency

2014-08-14 Thread Michael R. Hines
On 08/14/2014 06:58 PM, Dr. David Alan Gilbert wrote: cc'ing in a couple of the COLOers. Thanks, David. Glad to see their patches in last month - I need to take a look at them. The 2013 paper says: 'COLO modifies the guest OS’s TCP/IP stack in order to make the behavior more deterministic.

Re: [Qemu-devel] Microcheckpointing: Memory-VCPU / Disk State consistency

2014-08-13 Thread Walid Nouri
Yes... Time is a problem, and it‘s currently running out... ;-) I think the first step is to reason about possible approaches and how they can be implemented in QEMU. The implementation can follow later :-) Thank you for the hint with the drive-mirror feature. I will take a look at it and

Re: [Qemu-devel] Microcheckpointing: Memory-VCPU / Disk State consistency

2014-08-13 Thread Michael R. Hines
On 08/13/2014 10:03 PM, Walid Nouri wrote: While looking to find some ideas for approaches to replicating block devices I have read the paper about the Remus implementation. I think MC can take a similar approach for local disk. I agree. Here are the main facts that I have understood:

Re: [Qemu-devel] Microcheckpointing: Memory-VCPU / Disk State consistency

2014-08-11 Thread Michael R. Hines
Excellent question: QEMU does have a feature called drive-mirror in block/mirror.c that was introduced a couple of years ago. I'm not sure what the adoption rate of the feature is, but I would start with that one. There is also a second fault tolerance implementation that works a little

Re: [Qemu-devel] Microcheckpointing: Memory-VCPU / Disk State consistency

2014-08-11 Thread Michael R. Hines
Excellent question: QEMU does have a feature called drive-mirror in block/mirror.c that was introduced a couple of years ago. I'm not sure what the adoption rate of the feature is, but I would start with that one. There is also a second fault tolerance implementation that works a little