Re: [RFC PATCH 05/20] Introduce put_vector() and get_vector to QEMUFile and qemu_fopen_ops().

2010-04-26 Thread Yoshiaki Tamura
Anthony Liguori wrote: On 04/22/2010 10:37 PM, Yoshiaki Tamura wrote: Anthony Liguori wrote: On 04/21/2010 12:57 AM, Yoshiaki Tamura wrote: QEMUFile currently doesn't support writev(). For sending multiple data, such as pages, using writev() should be more efficient. Signed-off-by: Yoshiaki

Re: [Qemu-devel] [RFC PATCH 00/20] Kemari for KVM v0.1

2010-04-26 Thread Yoshiaki Tamura
Anthony Liguori wrote: On 04/22/2010 08:53 PM, Yoshiaki Tamura wrote: Anthony Liguori wrote: On 04/22/2010 08:16 AM, Yoshiaki Tamura wrote: 2010/4/22 Dor Laordl...@redhat.com: On 04/22/2010 01:35 PM, Yoshiaki Tamura wrote: Dor Laor wrote: On 04/21/2010 08:57 AM, Yoshiaki Tamura wrote: Hi

Re: [RFC PATCH 00/20] Kemari for KVM v0.1

2010-04-26 Thread Yoshiaki Tamura
Avi Kivity wrote: On 04/21/2010 08:57 AM, Yoshiaki Tamura wrote: Kemari starts synchronizing VMs when QEMU handles I/O requests. Without this patch VCPU state is already proceeded before synchronization, and after failover to the VM on the receiver, it hangs because of this. We discussed

Re: [PATCH 09/10] kvm: port qemu-kvm's bitmap scanning

2010-04-26 Thread Yoshiaki Tamura
Hi, This patch may conflict with the patch I posted on April 19. http://www.mail-archive.com/qemu-de...@nongnu.org/msg29941.html If Marcelo's is going to be merged, I need to rebase the above to it. It would be helpful if you could tell me the plan. Thanks, Yoshi 2010/4/27 Marcelo Tosatti

Re: [RFC PATCH 04/20] Make QEMUFile buf expandable, and introduce qemu_realloc_buffer() and qemu_clear_buffer().

2010-04-23 Thread Yoshiaki Tamura
Avi Kivity wrote: On 04/21/2010 08:57 AM, Yoshiaki Tamura wrote: Currently buf size is fixed at 32KB. It would be useful if it could be flexible. Why is this needed? The real buffering is in the kernel anyways; this is only used to reduce the number of write() syscalls. This was introduced

Re: [PATCH RFC v2 5/6] KVM: moving dirty bitmaps to user space

2010-04-23 Thread Yoshiaki Tamura
2010/4/23 Takuya Yoshikawa yoshikawa.tak...@oss.ntt.co.jp: (2010/04/23 19:28), Avi Kivity wrote: OK, I will do in the next version. In this RFC, I would be happy if I can know the overall design is right or not. Everything looks reasonable to me. Thank you! Do you have performance

Re: [Qemu-devel] [RFC PATCH 00/20] Kemari for KVM v0.1

2010-04-22 Thread Yoshiaki Tamura
Dor Laor wrote: On 04/21/2010 08:57 AM, Yoshiaki Tamura wrote: Hi all, We have been implementing the prototype of Kemari for KVM, and we're sending this message to share what we have now and TODO lists. Hopefully, we would like to get early feedback to keep us in the right direction. Although

Re: [Qemu-devel] [RFC PATCH 00/20] Kemari for KVM v0.1

2010-04-22 Thread Yoshiaki Tamura
2010/4/22 Takuya Yoshikawa yoshikawa.tak...@oss.ntt.co.jp: (2010/04/22 19:35), Yoshiaki Tamura wrote: A trivial one would we to : - do X online snapshots/sec I currently don't have good numbers that I can share right now. Snapshots/sec depends on what kind of workload is running

Re: [Qemu-devel] [RFC PATCH 00/20] Kemari for KVM v0.1

2010-04-22 Thread Yoshiaki Tamura
Jamie Lokier wrote: Yoshiaki Tamura wrote: Dor Laor wrote: On 04/21/2010 08:57 AM, Yoshiaki Tamura wrote: Event tapping is the core component of Kemari, and it decides on which event the primary should synchronize with the secondary. The basic assumption here is that outgoing I/O operations

Re: [RFC PATCH 00/20] Kemari for KVM v0.1

2010-04-22 Thread Yoshiaki Tamura
Anthony Liguori wrote: On 04/21/2010 12:57 AM, Yoshiaki Tamura wrote: Hi all, We have been implementing the prototype of Kemari for KVM, and we're sending this message to share what we have now and TODO lists. Hopefully, we would like to get early feedback to keep us in the right direction

Re: [Qemu-devel] [RFC PATCH 00/20] Kemari for KVM v0.1

2010-04-22 Thread Yoshiaki Tamura
Anthony Liguori wrote: On 04/22/2010 08:16 AM, Yoshiaki Tamura wrote: 2010/4/22 Dor Laordl...@redhat.com: On 04/22/2010 01:35 PM, Yoshiaki Tamura wrote: Dor Laor wrote: On 04/21/2010 08:57 AM, Yoshiaki Tamura wrote: Hi all, We have been implementing the prototype of Kemari for KVM

Re: [RFC PATCH 01/20] Modify DIRTY_FLAG value and introduce DIRTY_IDX to use as indexes of bit-based phys_ram_dirty.

2010-04-22 Thread Yoshiaki Tamura
Anthony Liguori wrote: Hi, On 04/21/2010 12:57 AM, Yoshiaki Tamura wrote: Replaces byte-based phys_ram_dirty bitmap with four (MASTER, VGA, CODE, MIGRATION) bit-based phys_ram_dirty bitmap. On allocation, it sets all bits in the bitmap. It uses ffs() to convert DIRTY_FLAG to DIRTY_IDX

Re: [RFC PATCH 15/20] Introduce FT mode support to configure.

2010-04-22 Thread Yoshiaki Tamura
Anthony Liguori wrote: On 04/21/2010 12:57 AM, Yoshiaki Tamura wrote: Signed-off-by: Yoshiaki Tamuratamura.yoshi...@lab.ntt.co.jp No need for this. OK. Regards, Anthony Liguori --- configure | 8 1 files changed, 8 insertions(+), 0 deletions(-) diff --git a/configure b

Re: [RFC PATCH 14/20] Upgrade QEMU_FILE_VERSION from 3 to 4, and introduce qemu_savevm_state_all().

2010-04-22 Thread Yoshiaki Tamura
Anthony Liguori wrote: On 04/21/2010 12:57 AM, Yoshiaki Tamura wrote: Make a 32bit entry after QEMU_VM_FILE_VERSION to recognize whether the transfered data is QEMU_VM_FT_MODE or QEMU_VM_LIVE_MIGRATION_MODE. I'd rather you encapsulate the current protocol as opposed to extending it with a new

Re: [RFC PATCH 05/20] Introduce put_vector() and get_vector to QEMUFile and qemu_fopen_ops().

2010-04-22 Thread Yoshiaki Tamura
Anthony Liguori wrote: On 04/21/2010 12:57 AM, Yoshiaki Tamura wrote: QEMUFile currently doesn't support writev(). For sending multiple data, such as pages, using writev() should be more efficient. Signed-off-by: Yoshiaki Tamuratamura.yoshi...@lab.ntt.co.jp Is there performance data

Re: [RFC PATCH 07/20] Introduce qemu_put_vector() and qemu_put_vector_prepare() to use put_vector() in QEMUFile.

2010-04-22 Thread Yoshiaki Tamura
Anthony Liguori wrote: On 04/21/2010 12:57 AM, Yoshiaki Tamura wrote: For fool proof purpose, qemu_put_vector_parepare should be called before qemu_put_vector. Then, if qemu_put_* functions except this is called after qemu_put_vector_prepare, program will abort(). Signed-off-by: Yoshiaki

Re: [RFC PATCH 10/20] Introduce skip_header parameter to qemu_loadvm_state() so that it can be called iteratively without reading the header.

2010-04-22 Thread Yoshiaki Tamura
Anthony Liguori wrote: On 04/21/2010 12:57 AM, Yoshiaki Tamura wrote: Signed-off-by: Yoshiaki Tamuratamura.yoshi...@lab.ntt.co.jp I think the more appropriate thing to do is have qemu_savevm_state_complete() not write QEMU_VM_EOF when doing a continuous live migration. You would then want

Re: [RFC PATCH 19/20] Insert do_event_tap() to virtio-{blk, net}, comment out assert() on cpu_single_env temporally.

2010-04-22 Thread Yoshiaki Tamura
Anthony Liguori wrote: On 04/21/2010 12:57 AM, Yoshiaki Tamura wrote: do_event_tap() is inserted to functions which actually fire outputs. By synchronizing VMs before outputs are fired, we can failover to the receiver upon failure. To save VM continuously, comment out assert() on cpu_single_env

Re: [Qemu-devel] [RFC PATCH 00/20] Kemari for KVM v0.1

2010-04-22 Thread Yoshiaki Tamura
Dor Laor wrote: On 04/22/2010 04:16 PM, Yoshiaki Tamura wrote: 2010/4/22 Dor Laordl...@redhat.com: On 04/22/2010 01:35 PM, Yoshiaki Tamura wrote: Dor Laor wrote: On 04/21/2010 08:57 AM, Yoshiaki Tamura wrote: Hi all, We have been implementing the prototype of Kemari for KVM, and we're

[RFC PATCH 07/20] Introduce qemu_put_vector() and qemu_put_vector_prepare() to use put_vector() in QEMUFile.

2010-04-21 Thread Yoshiaki Tamura
For fool proof purpose, qemu_put_vector_parepare should be called before qemu_put_vector. Then, if qemu_put_* functions except this is called after qemu_put_vector_prepare, program will abort(). Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp --- hw/hw.h |2 ++ savevm.c | 53

[RFC PATCH 16/20] Introduce event_tap fucntions and ft_tranx_ready().

2010-04-21 Thread Yoshiaki Tamura
event_tap controls when to start ft transaction. do_event_tap() should be instered to the device emulators. ft_tranx_ready() kicks the transaction. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp --- migration.c | 77

[RFC PATCH 10/20] Introduce skip_header parameter to qemu_loadvm_state() so that it can be called iteratively without reading the header.

2010-04-21 Thread Yoshiaki Tamura
Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp --- migration-exec.c |2 +- migration-fd.c |2 +- migration-tcp.c |2 +- migration-unix.c |2 +- savevm.c | 25 ++--- sysemu.h |2 +- 6 files changed, 19 insertions(+), 16

[RFC PATCH 11/20] Introduce some socket util functions.

2010-04-21 Thread Yoshiaki Tamura
Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp --- osdep.c | 13 + qemu-char.c | 25 - qemu_socket.h |4 3 files changed, 41 insertions(+), 1 deletions(-) diff --git a/osdep.c b/osdep.c index 3bab79a..63444e7 100644

[RFC PATCH 06/20] Introduce iovec util functions, qemu_iovec_to_vector() and qemu_iovec_to_size().

2010-04-21 Thread Yoshiaki Tamura
Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp --- cutils.c | 12 qemu-common.h |2 ++ 2 files changed, 14 insertions(+), 0 deletions(-) diff --git a/cutils.c b/cutils.c index be99b21..1d35590 100644 --- a/cutils.c +++ b/cutils.c @@ -238,3 +238,15 @@ void

[RFC PATCH 13/20] Introduce util functions to control ft_transaction from savevm layer.

2010-04-21 Thread Yoshiaki Tamura
To utilize ft_transaction function, savevm needs interfaces to be exported. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp --- hw/hw.h |5 + savevm.c | 41 + 2 files changed, 46 insertions(+), 0 deletions(-) diff --git a/hw/hw.h b

[RFC PATCH 12/20] Introduce fault tolerant VM transaction QEMUFile and ft_mode.

2010-04-21 Thread Yoshiaki Tamura
This code implements VM transaction protocol. Like buffered_file, it sits between savevm and migration layer. With this architecture, VM transaction protocol is implemented mostly independent from other existing code. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off

[RFC PATCH 05/20] Introduce put_vector() and get_vector to QEMUFile and qemu_fopen_ops().

2010-04-21 Thread Yoshiaki Tamura
QEMUFile currently doesn't support writev(). For sending multiple data, such as pages, using writev() should be more efficient. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp --- buffered_file.c |2 +- hw/hw.h | 16 savevm.c| 43

[RFC PATCH 19/20] Insert do_event_tap() to virtio-{blk,net}, comment out assert() on cpu_single_env temporally.

2010-04-21 Thread Yoshiaki Tamura
do_event_tap() is inserted to functions which actually fire outputs. By synchronizing VMs before outputs are fired, we can failover to the receiver upon failure. To save VM continuously, comment out assert() on cpu_single_env temporally. Signed-off-by: Yoshiaki Tamura tamura.yoshi

[RFC PATCH 20/20] Introduce -k option to enable FT migration mode (Kemari).

2010-04-21 Thread Yoshiaki Tamura
When -k option is set to migrate command, it will turn on ft_mode to start FT migration mode (Kemari). Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp --- migration.c |3 +++ qemu-monitor.hx |7 --- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git

[RFC PATCH 01/20] Modify DIRTY_FLAG value and introduce DIRTY_IDX to use as indexes of bit-based phys_ram_dirty.

2010-04-21 Thread Yoshiaki Tamura
phys_ram_dirty bitmap. MASTER works as a buffer, and upon get_diry() or get_dirty_flags(), it calls cpu_physical_memory_sync_master() to update VGA and MIGRATION. Replaces direct phys_ram_dirty access with wrapper functions to prevent direct access to the phys_ram_dirty bitmap. Signed-off-by: Yoshiaki

[RFC PATCH 03/20] Use cpu_physical_memory_set_dirty_range() to update phys_ram_dirty.

2010-04-21 Thread Yoshiaki Tamura
Modifies kvm_get_dirty_pages_log_range to use cpu_physical_memory_set_dirty_range() to update the row of the bit-based phys_ram_dirty bitmap at once. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- qemu-kvm.c | 19

[RFC PATCH 00/20] Kemari for KVM v0.1

2010-04-21 Thread Yoshiaki Tamura
. Without this patch VCPU state is already proceeded before synchronization, and after failover to the VM on the receiver, it hangs because of this. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp --- arch/x86/include/asm/kvm_host.h |1 + arch/x86/kvm/svm.c | 11

[RFC PATCH 02/20] Introduce cpu_physical_memory_get_dirty_range().

2010-04-21 Thread Yoshiaki Tamura
It checks the first row and puts dirty addr in the array. If the first row is empty, it skips to the first non-dirty row or the end addr, and put the length in the first entry of the array. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off-by: OHMURA Kei ohmura

[RFC PATCH 17/20] Modify migrate_fd_put_ready() when ft_mode is on.

2010-04-21 Thread Yoshiaki Tamura
When ft_mode is on, migrate_fd_put_ready() would open ft_transaction file and turn on event_tap. To end or cancel ft_transaction, ft_mode and event_tap is turned off. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp --- migration.c | 36 +--- 1

[RFC PATCH 04/20] Make QEMUFile buf expandable, and introduce qemu_realloc_buffer() and qemu_clear_buffer().

2010-04-21 Thread Yoshiaki Tamura
Currently buf size is fixed at 32KB. It would be useful if it could be flexible. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp --- hw/hw.h |2 ++ savevm.c | 26 +- 2 files changed, 27 insertions(+), 1 deletions(-) diff --git a/hw/hw.h b/hw/hw.h

[RFC PATCH 09/20] Introduce writev and read to FdMigrationState.

2010-04-21 Thread Yoshiaki Tamura
Currently FdMigrationState doesn't support writev() and read(). writev() is introduced to send data efficiently, and read() is necessary to get response from the other side. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp --- migration-tcp.c | 20 migration.c

[RFC PATCH 15/20] Introduce FT mode support to configure.

2010-04-21 Thread Yoshiaki Tamura
Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp --- configure |8 1 files changed, 8 insertions(+), 0 deletions(-) diff --git a/configure b/configure index 046c591..f0682d4 100755 --- a/configure +++ b/configure @@ -298,6 +298,7 @@ bsd_user=no guest_base

[RFC PATCH 08/20] Introduce RAMSaveIO and use cpu_physical_memory_get_dirty_range() to check multiple dirty pages.

2010-04-21 Thread Yoshiaki Tamura
Introduce RAMSaveIO to use writev for saving ram blocks, and modifies ram_save_block() and ram_save_remaining() to use cpu_physical_memory_get_dirty_range() to check multiple dirty and non-dirty pages at once. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off-by: OHMURA Kei

Re: [RFC PATCH 04/20] Make QEMUFile buf expandable, and introduce qemu_realloc_buffer() and qemu_clear_buffer().

2010-04-21 Thread Yoshiaki Tamura
2010/4/21 Stefan Hajnoczi stefa...@gmail.com: On Wed, Apr 21, 2010 at 6:57 AM, Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp wrote: @@ -454,6 +458,25 @@ void qemu_fflush(QEMUFile *f)     }  } +void *qemu_realloc_buffer(QEMUFile *f, int size) +{ +    f-buf_max_size = size; + +    f-buf

Re: [PATCH v2 3/6] Modifies wrapper functions for byte-based phys_ram_dirty bitmap to bit-based phys_ram_dirty bitmap.

2010-04-13 Thread Yoshiaki Tamura
Avi Kivity wrote: On 04/12/2010 01:58 PM, Yoshiaki Tamura wrote: Is it necessary to update migration and vga bitmaps? We can simply update the master bitmap, and update the migration and vga bitmaps only when they need it. That can be done in a different patch. Let me explain the role

Re: [PATCH v2 3/6] Modifies wrapper functions for byte-based phys_ram_dirty bitmap to bit-based phys_ram_dirty bitmap.

2010-04-13 Thread Yoshiaki Tamura
Avi Kivity wrote: On 04/13/2010 11:01 AM, Yoshiaki Tamura wrote: Avi Kivity wrote: On 04/12/2010 01:58 PM, Yoshiaki Tamura wrote: Is it necessary to update migration and vga bitmaps? We can simply update the master bitmap, and update the migration and vga bitmaps only when they need

Re: [PATCH v2 2/6] Introduce bit-based phys_ram_dirty for VGA, CODE, MIGRATION and MASTER.

2010-04-12 Thread Yoshiaki Tamura
Avi Kivity wrote: On 04/06/2010 03:51 AM, Yoshiaki Tamura wrote: Replaces byte-based phys_ram_dirty bitmap with three bit-based phys_ram_dirty bitmap. On allocation, it sets all bits in the bitmap. index c74b0a4..9733892 100644 --- a/exec.c +++ b/exec.c @@ -110,7 +110,7 @@ uint8_t

Re: [PATCH v2 3/6] Modifies wrapper functions for byte-based phys_ram_dirty bitmap to bit-based phys_ram_dirty bitmap.

2010-04-12 Thread Yoshiaki Tamura
Avi Kivity wrote: On 04/06/2010 03:51 AM, Yoshiaki Tamura wrote: Signed-off-by: Yoshiaki Tamuratamura.yoshi...@lab.ntt.co.jp Signed-off-by: OHMURA Keiohmura@lab.ntt.co.jp --- static inline int cpu_physical_memory_get_dirty_flags(ram_addr_t addr) { - return phys_ram_dirty[addr

Re: Question on skip_emulated_instructions()

2010-04-08 Thread Yoshiaki Tamura
Gleb Natapov wrote: On Thu, Apr 08, 2010 at 02:27:53PM +0900, Yoshiaki Tamura wrote: Avi Kivity wrote: On 04/07/2010 08:21 PM, Yoshiaki Tamura wrote: The problem here is that, I needed to transfer the VM state which is just *before* the output to the devices. Otherwise, the VM state has

Re: Question on skip_emulated_instructions()

2010-04-08 Thread Yoshiaki Tamura
Gleb Natapov wrote: On Thu, Apr 08, 2010 at 02:27:53PM +0900, Yoshiaki Tamura wrote: Currently we complete instructions for output operations and leave them incomplete for input operations. Deferring completion for output operations should work, except it may break the vmware backdoor port (see

Re: Question on skip_emulated_instructions()

2010-04-08 Thread Yoshiaki Tamura
Gleb Natapov wrote: On Thu, Apr 08, 2010 at 10:17:01AM +0300, Avi Kivity wrote: On 04/08/2010 08:27 AM, Yoshiaki Tamura wrote: The requirement is that the guest must always be able to replay at least the instruction which triggered the synchronization on the primary. You have two choices

Re: Question on skip_emulated_instructions()

2010-04-08 Thread Yoshiaki Tamura
Avi Kivity wrote: On 04/08/2010 10:30 AM, Yoshiaki Tamura wrote: To answer your question, it should be possible to implement. The down side is that after going into KVM to make the guest state to consistent, we need to go back to qemu to actually transfer the guest, and this bounce would

Re: Question on skip_emulated_instructions()

2010-04-08 Thread Yoshiaki Tamura
Avi Kivity wrote: On 04/08/2010 11:30 AM, Yoshiaki Tamura wrote: If I transferred a VM after I/O operations, let's say the VM sent an TCP ACK to the client, and if a hardware failure occurred to the primary during the VM transferring *but the client received the TCP ACK*, the secondary

Re: Question on skip_emulated_instructions()

2010-04-08 Thread Yoshiaki Tamura
2010/4/8 Avi Kivity a...@redhat.com: On 04/08/2010 12:14 PM, Yoshiaki Tamura wrote: I don't think you can in the general case. But if you gate output at the device level, instead of the instruction level, the problem goes away, no? Yes, it should. To implement this, we need to make No.3

Re: Question on skip_emulated_instructions()

2010-04-07 Thread Yoshiaki Tamura
2010/4/6 Gleb Natapov g...@redhat.com: On Tue, Apr 06, 2010 at 01:11:23PM +0900, Yoshiaki Tamura wrote: Hi. When handle_io() is called, rip is currently proceeded *before* actually having I/O handled by qemu in userland.  Upon implementing Kemari for KVM(http://www.mail-archive.com/kvm

Re: Question on skip_emulated_instructions()

2010-04-07 Thread Yoshiaki Tamura
2010/4/8 Gleb Natapov g...@redhat.com: On Wed, Apr 07, 2010 at 03:25:10PM +0900, Yoshiaki Tamura wrote: 2010/4/6 Gleb Natapov g...@redhat.com: On Tue, Apr 06, 2010 at 01:11:23PM +0900, Yoshiaki Tamura wrote: Hi. When handle_io() is called, rip is currently proceeded *before* actually

Re: Question on skip_emulated_instructions()

2010-04-07 Thread Yoshiaki Tamura
Avi Kivity wrote: On 04/07/2010 08:21 PM, Yoshiaki Tamura wrote: The problem here is that, I needed to transfer the VM state which is just *before* the output to the devices. Otherwise, the VM state has already been proceeded, and after failover, some I/O didn't work as I expected. I tracked

[PATCH v2 0/6] Introduce bit-based phys_ram_dirty, and bit-based dirty page checker.

2010-04-05 Thread Yoshiaki Tamura
Yoshiaki Tamura (6): Modify DIRTY_FLAG value to use as indexes of bit-based phys_ram_dirty. Introduce bit-based phys_ram_dirty for VGA, CODE, MIGRATION and MASTER. Modifies wrapper functions for byte-based phys_ram_dirty bitmap to bit-based phys_ram_dirty bitmap. Introduce

[PATCH v2 1/6] Modify DIRTY_FLAG value to use as indexes of bit-based phys_ram_dirty.

2010-04-05 Thread Yoshiaki Tamura
Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- cpu-all.h | 10 -- 1 files changed, 8 insertions(+), 2 deletions(-) diff --git a/cpu-all.h b/cpu-all.h index f8bfa66..c409fad 100644 --- a/cpu-all.h +++ b/cpu-all.h

[PATCH v2 4/6] Introduce cpu_physical_memory_get_dirty_range().

2010-04-05 Thread Yoshiaki Tamura
Introduces cpu_physical_memory_get_dirty_range(). It checks the first row and puts dirty addr in the array. If the first row is empty, it skips to the first non-dirty row or the end addr, and put the length in the first entry of the array. Signed-off-by: Yoshiaki Tamura tamura.yoshi

[PATCH v2 3/6] Modifies wrapper functions for byte-based phys_ram_dirty bitmap to bit-based phys_ram_dirty bitmap.

2010-04-05 Thread Yoshiaki Tamura
Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- cpu-all.h | 81 ++-- 1 files changed, 67 insertions(+), 14 deletions(-) diff --git a/cpu-all.h b/cpu-all.h index c409fad

[PATCH v2 2/6] Introduce bit-based phys_ram_dirty for VGA, CODE, MIGRATION and MASTER.

2010-04-05 Thread Yoshiaki Tamura
Replaces byte-based phys_ram_dirty bitmap with three bit-based phys_ram_dirty bitmap. On allocation, it sets all bits in the bitmap. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- exec.c | 32

[PATCH v2 5/6] Use cpu_physical_memory_set_dirty_range() to update phys_ram_dirty.

2010-04-05 Thread Yoshiaki Tamura
Modifies kvm_physical_sync_dirty_bitmap to use cpu_physical_memory_set_dirty_range() to update the row of the bit-based phys_ram_dirty bitmap at once. Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp --- bswap.h |2 ++ kvm

[PATCH v2 6/6] Use cpu_physical_memory_get_dirty_range() to check multiple dirty pages.

2010-04-05 Thread Yoshiaki Tamura
Modifies ram_save_block() and ram_save_remaining() to use cpu_physical_memory_get_dirty_range() to check multiple dirty and non-dirty pages at once. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- arch_init.c | 54

Question on skip_emulated_instructions()

2010-04-05 Thread Yoshiaki Tamura
Hi. When handle_io() is called, rip is currently proceeded *before* actually having I/O handled by qemu in userland. Upon implementing Kemari for KVM(http://www.mail-archive.com/kvm@vger.kernel.org/msg25141.html) mainly in userland qemu, we encountered a problem that synchronizing the content of

Re: [PATCH 0/2] qemu-kvm: Introduce wrapper functions to access phys_ram_dirty, and replace existing direct accesses to it.

2010-03-21 Thread Yoshiaki Tamura
Marcelo Tosatti wrote: On Wed, Mar 17, 2010 at 02:51:46PM +0900, Yoshiaki Tamura wrote: Before replacing byte-based dirty bitmap with bit-based dirty bitmap, clearing direct accesses to the bitmap first seems to be good point to start with. This patch set is based on the following discussion

[PATCH 0/6] qemu-kvm: Introduce bit-based phys_ram_dirty, and bit-based dirty page checker.

2010-03-16 Thread Yoshiaki Tamura
The dirty and non-dirty pages are checked one by one in vl.c. When the most of the memory is not dirty, checking the dirty and non-dirty pages by multiple page size should be much faster than checking them one by one. We introduced bit-based phys_ram_dirty for VGA, CODE and MIGRATION, and

[PATCH 4/6] qemu-kvm: Introduce cpu_physical_memory_get_dirty_range().

2010-03-16 Thread Yoshiaki Tamura
Introduces cpu_physical_memory_get_dirty_range(). It checks the first row and puts dirty addr in the array. If the first row is empty, it skips to the first non-dirty row or the end addr, and put the length in the first entry of the array. Signed-off-by: Yoshiaki Tamura tamura.yoshi

[PATCH 2/6] qemu-kvm: Modify and introduce wrapper functions to access phys_ram_dirty.

2010-03-16 Thread Yoshiaki Tamura
Modifies wrapper functions for byte-based phys_ram_dirty bitmap to bit-based phys_ram_dirty bitmap, and adds more wrapper functions to prevent direct access to the phys_ram_dirty bitmap. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off-by: OHMURA Kei ohmura

[PATCH 3/6] qemu-kvm: Replace direct phys_ram_dirty access with wrapper functions.

2010-03-16 Thread Yoshiaki Tamura
Replaces direct phys_ram_dirty access with wrapper functions to prevent direct access to the phys_ram_dirty bitmap. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- exec.c | 45 - 1

[PATCH 6/6] qemu-kvm: Use cpu_physical_memory_get_dirty_range() to check multiple dirty pages.

2010-03-16 Thread Yoshiaki Tamura
Modifies ram_save_block() and ram_save_remaining() to use cpu_physical_memory_get_dirty_range() to check multiple dirty and non-dirty pages at once. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- vl.c | 55

[PATCH 1/6] qemu-kvm: Introduce bit-based phys_ram_dirty for VGA, CODE and MIGRATION.

2010-03-16 Thread Yoshiaki Tamura
Replaces byte-based phys_ram_dirty bitmap with three bit-based phys_ram_dirty bitmap. On allocation, it sets all bits in the bitmap. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- exec.c | 22 +- 1 files

[PATCH 5/6] qemu-kvm: Use cpu_physical_memory_set_dirty_range() to update phys_ram_dirty.

2010-03-16 Thread Yoshiaki Tamura
Modifies kvm_get_dirty_pages_log_range to use cpu_physical_memory_set_dirty_range() to update the row of the bit-based phys_ram_dirty bitmap at once. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- qemu-kvm.c | 19

Re: [PATCH 1/6] qemu-kvm: Introduce bit-based phys_ram_dirty for VGA, CODE and MIGRATION.

2010-03-16 Thread Yoshiaki Tamura
Avi Kivity wrote: On 03/16/2010 12:53 PM, Yoshiaki Tamura wrote: Replaces byte-based phys_ram_dirty bitmap with three bit-based phys_ram_dirty bitmap. On allocation, it sets all bits in the bitmap. Signed-off-by: Yoshiaki Tamuratamura.yoshi...@lab.ntt.co.jp Signed-off-by: OHMURA Keiohmura

Re: [PATCH 2/6] qemu-kvm: Modify and introduce wrapper functions to access phys_ram_dirty.

2010-03-16 Thread Yoshiaki Tamura
Avi Kivity wrote: On 03/16/2010 12:53 PM, Yoshiaki Tamura wrote: Modifies wrapper functions for byte-based phys_ram_dirty bitmap to bit-based phys_ram_dirty bitmap, and adds more wrapper functions to prevent direct access to the phys_ram_dirty bitmap. + +static inline int

Re: [PATCH 0/6] qemu-kvm: Introduce bit-based phys_ram_dirty, and bit-based dirty page checker.

2010-03-16 Thread Yoshiaki Tamura
Avi Kivity wrote: On 03/16/2010 12:53 PM, Yoshiaki Tamura wrote: Experimental results: Cond1: 1.9 ~ 61 times speed up Cond2: 1.9 ~ 56 times speed up Cond3: 1.9 ~ 59 times speed up Cond4: 1.7 ~ 59 times speed up Impressive results. What's the typical speedup? Closer to 1.9 or 61

Re: [PATCH 2/6] qemu-kvm: Modify and introduce wrapper functions to access phys_ram_dirty.

2010-03-16 Thread Yoshiaki Tamura
Avi Kivity wrote: On 03/16/2010 03:17 PM, Yoshiaki Tamura wrote: Avi Kivity wrote: On 03/16/2010 12:53 PM, Yoshiaki Tamura wrote: Modifies wrapper functions for byte-based phys_ram_dirty bitmap to bit-based phys_ram_dirty bitmap, and adds more wrapper functions to prevent direct access

Re: [PATCH 2/6] qemu-kvm: Modify and introduce wrapper functions to access phys_ram_dirty.

2010-03-16 Thread Yoshiaki Tamura
Anthony Liguori wrote: On 03/16/2010 07:45 AM, Avi Kivity wrote: On 03/16/2010 12:53 PM, Yoshiaki Tamura wrote: Modifies wrapper functions for byte-based phys_ram_dirty bitmap to bit-based phys_ram_dirty bitmap, and adds more wrapper functions to prevent direct access to the phys_ram_dirty

[PATCH 0/2] qemu-kvm: Introduce wrapper functions to access phys_ram_dirty, and replace existing direct accesses to it.

2010-03-16 Thread Yoshiaki Tamura
Before replacing byte-based dirty bitmap with bit-based dirty bitmap, clearing direct accesses to the bitmap first seems to be good point to start with. This patch set is based on the following discussion. http://www.mail-archive.com/kvm@vger.kernel.org/msg30724.html Thanks, Yoshi -- To

[PATCH 1/2] qemu-kvm: Introduce wrapper functions to access phys_ram_dirty.

2010-03-16 Thread Yoshiaki Tamura
Adds wrapper functions to prevent direct access to the phys_ram_dirty bitmap. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- cpu-all.h | 25 + 1 files changed, 25 insertions(+), 0 deletions(-) diff

[PATCH 2/2] qemu-kvm: Replace direct phys_ram_dirty access with wrapper functions.

2010-03-16 Thread Yoshiaki Tamura
Replaces direct phys_ram_dirty access with wrapper functions to prevent direct access to the phys_ram_dirty bitmap. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- exec.c | 45 - 1

Re: [PATCH 1/3] qemu-kvm: Wrap phys_ram_dirty with additional inline functions.

2010-02-14 Thread Yoshiaki Tamura
Avi Kivity wrote: On 02/12/2010 04:08 AM, OHMURA Kei wrote: Why do you need a counter? It may be sufficient to set a single bit. This reduces the memory overhead and perhaps cache thrashing. Thanks for looking into this. I agree with your opinion. Our motivation here is to skip traveling when

Re: [RFC] KVM Fault Tolerance: Kemari for KVM

2009-11-18 Thread Yoshiaki Tamura
2009/11/17 Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp: Avi Kivity wrote: On 11/16/2009 04:18 PM, Fernando Luis Vázquez Cao wrote: Avi Kivity wrote: On 11/09/2009 05:53 AM, Fernando Luis Vázquez Cao wrote: Kemari runs paired virtual machines in an active-passive configuration

Re: [RFC] KVM Fault Tolerance: Kemari for KVM

2009-11-18 Thread Yoshiaki Tamura
2009/11/18 Avi Kivity a...@redhat.com: On 11/18/2009 03:28 PM, Yoshiaki Tamura wrote: I don't think lmbench is intensive but it's sensitive to memory latency. We'll measure kernel build time with minimum config, and post it later. Here are some quick numbers of parallel kernel compile time

Re: [RFC] KVM Fault Tolerance: Kemari for KVM

2009-11-17 Thread Yoshiaki Tamura
Avi Kivity wrote: On 11/16/2009 04:18 PM, Fernando Luis Vázquez Cao wrote: Avi Kivity wrote: On 11/09/2009 05:53 AM, Fernando Luis Vázquez Cao wrote: Kemari runs paired virtual machines in an active-passive configuration and achieves whole-system replication by continuously copying the state

Re: [RFC] KVM Fault Tolerance: Kemari for KVM

2009-11-17 Thread Yoshiaki Tamura
2009/11/17 Avi Kivity a...@redhat.com: On 11/17/2009 01:04 PM, Yoshiaki Tamura wrote: What I mean is: - choose synchronization point A - start copying memory for synchronization point A  - output is delayed - choose synchronization point B - copy memory for A and B   if guest touches

<    1   2   3   4   5