Re: [KVM][Kemari]: Build error fix

On 2012/06/28 5:34, Sterling Windmill wrote: Is Kemari still in active development? No, it's not. Currently we have no intention to add new features into Kemari. Thanks, Kei Best regards, Sterling Windmill On Sun, Dec 4, 2011 at 9:45 PM, OHMURA Kei ohmura@lab.ntt.co.jp mailto:ohmura

Re: [KVM][Kemari]: Build error fix

On 2011/12/02 21:51, Pradeep Kumar wrote: It fixes build failure. I hit this error, after succsfull migration and sync. (qemu) qemu-system-x86_64: fill buffer failed, Interrupted system call qemu-system-x86_64: recv header failed qemu-system-x86_64: recv ack failed

[PATCH 13/18] net: insert event-tap to qemu_send_packet() and qemu_sendv_packet_async().

From: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp event-tap function is called only when it is on. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- net.c |9 + 1 files changed, 9 insertions(+), 0 deletions(-) diff

[PATCH 04/18] qemu-char: export socket_set_nodelay().

From: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- qemu-char.c |2 +- qemu_socket.h |1 + 2 files changed, 2 insertions(+), 1 deletions(-) diff --git a/qemu-char.c b

[PATCH 02/18] Introduce read() to FdMigrationState.

...@lab.ntt.co.jp Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- migration-tcp.c | 15 +++ migration.c | 13 + migration.h |3 +++ 3 files changed, 31 insertions(+), 0 deletions(-) diff --git a/migration-tcp.c b/migration-tcp.c index d3d80c9..bb67d53

[PATCH 08/18] savevm: introduce util functions to control ft_trans_file from savevm layer.

From: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp To utilize ft_trans_file function, savevm needs interfaces to be exported. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- hw/hw.h |5 ++ savevm.c | 150

[PATCH 01/18] Make QEMUFile buf expandable, and introduce qemu_realloc_buffer() and qemu_clear_buffer().

From: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Currently buf size is fixed at 32KB. It would be useful if it could be flexible. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- hw/hw.h |2 ++ savevm.c | 20

[PATCH 16/18] migration: introduce migrate_ft_trans_{put,get}_ready(), and modify migrate_fd_put_ready() when ft_mode is on.

. migrate_ft_trans_get_ready() is called to receive ack from the receiver. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- migration.c | 266 ++- 1 files changed, 265 insertions

[PATCH 03/18] Introduce qemu_loadvm_state_no_header() and make qemu_loadvm_state() a wrapper.

From: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Introduce qemu_loadvm_state_no_header() so that it can be called iteratively without reading the header, and qemu_loadvm_state() becomes a wrapper of it. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off-by: OHMURA Kei

[PATCH 06/18] virtio: decrement last_avail_idx with inuse before saving.

From: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp For regular migration inuse == 0 always as requests are flushed before save. However, event-tap log when enabled introduces an extra queue for requests which is not being flushed, thus the last inuse requests are left in the event-tap queue.

[PATCH 05/18] vl.c: add deleted flag for deleting the handler.

From: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Make deleting handlers robust against deletion of any elements in a handler by using a deleted flag like in file descriptors. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp

[PATCH 18/18] Introduce kemari: to enable FT migration mode (Kemari).

tamura.yoshi...@lab.ntt.co.jp Acked-by: Paolo Bonzini pbonz...@redhat.com Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- hmp-commands.hx |4 +++- migration.c | 12 qmp-commands.hx |4 +++- 3 files changed, 18 insertions(+), 2 deletions(-) diff --git a/hmp-commands.hx b/hmp

[PATCH 15/18] savevm: introduce qemu_savevm_trans_{begin,commit}.

-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- savevm.c | 157 +++--- sysemu.h |2 + 2 files changed, 101 insertions(+), 58 deletions(-) diff --git a/savevm.c b/savevm.c index 5b57e94

[PATCH 10/18] Call init handler of event-tap at main() in vl.c.

From: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- vl.c |3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/vl.c b/vl.c index a18792d..2dbda4d 100644

[PATCH 11/18] ioport: insert event_tap_ioport() to ioport_write().

From: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Record ioport event to replay it upon failover. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- ioport.c |2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff

[PATCH 14/18] block: insert event-tap to bdrv_aio_writev(), bdrv_aio_flush() and bdrv_flush().

From: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp event-tap function is called only when it is on, and requests were sent from device emulators. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Acked-by: Kevin Wolf kw...@redhat.com Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp

[PATCH 09/18] Introduce event-tap.

event-tap controls when to start FT transaction, and provides proxy functions to called from net/block devices. While FT transaction, it queues up net/block requests, and flush them when the transaction gets completed. Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp Signed-off-by: Yoshiaki

[PATCH 12/18] Insert event_tap_mmio() to cpu_physical_memory_rw() in exec.c.

From: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Record mmio write event to replay it upon failover. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- exec.c |4 1 files changed, 4 insertions(+), 0 deletions(-) diff

[PATCH 00/18] Kemari for KVM v0.2.14

. git://kemari.git.sourceforge.net/gitroot/kemari/kemari next Thanks, Kei OHMURA Kei (2): Introduce fault tolerant VM transaction QEMUFile and ft_mode. Introduce event-tap. Yoshiaki Tamura (16): Make QEMUFile buf expandable, and introduce qemu_realloc_buffer() and qemu_clear_buffer

[PATCH 07/18] Introduce fault tolerant VM transaction QEMUFile and ft_mode.

This code implements VM transaction protocol. Like buffered_file, it sits between savevm and migration layer. With this architecture, VM transaction protocol is implemented mostly independent from other existing code. Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp Signed-off-by: Yoshiaki

[PATCH 17/18] migration-tcp: modify tcp_accept_incoming_migration() to handle ft_mode, and add a hack not to close fd when ft_mode is enabled.

mode, so that we can reuse the fd for it. vm_change_state_handler is added to turn off ft_mode when cont is pressed. Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- migration-tcp.c | 68

Re: virtio_blk_load() question

Thanks for your reply. When I ported virtio to vmstate, I was unable to get that list not empty for more than I tried. It should be not empty in the case of one error or similar, but I was not able to reproduce it. Actually, I wasn't able to get that condition either. We're having problem in

Re: virtio_blk_load() question

On 2010/03/18 21:07, Juan Quintela wrote: Really, ordering doesn't matter (in this case): see virtio-blk.c:virtio_blk_dma_restart_bh() QLIST_FOREACH_SAFE(req, rq_copy, next, next_req) { QLIST_REMOVE(req, next); virtio_blk_handle_request(req, mrb); } This mean that we

Re: [Qemu-devel] Re: [PATCH v2] qemu-kvm: Speed up of the dirty-bitmap-traveling

We think? I mean - yes, I think so too. But have you actually measured it? How much improvement are we talking here? Is it still faster when a bswap is involved? Thanks for pointing out. I will post the data for x86 later. However, I don't have a test environment to check the impact of bswap.

Re: [Qemu-devel] Re: [PATCH v2] qemu-kvm: Speed up of the dirty-bitmap-traveling

We think? I mean - yes, I think so too. But have you actually measured it? How much improvement are we talking here? Is it still faster when a bswap is involved? Thanks for pointing out. I will post the data for x86 later. However, I don't have a test environment to check the impact of bswap.

Re: [Qemu-devel] Re: [PATCH v2] qemu-kvm: Speed up of the dirty-bitmap-traveling

We think? I mean - yes, I think so too. But have you actually measured it? How much improvement are we talking here? Is it still faster when a bswap is involved? Thanks for pointing out. I will post the data for x86 later. However, I don't have a test environment to check the impact of bswap.

Re: [PATCH v2] qemu-kvm: Speed up of the dirty-bitmap-traveling

dirty-bitmap-traveling is carried out by byte size in qemu-kvm.c. But We think that dirty-bitmap-traveling by long size is faster than by byte size especially when most of memory is not dirty. Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- bswap.h|2 ++ qemu-kvm.c | 31

Re: [PATCH v2] qemu-kvm: Speed up of the dirty-bitmap-traveling

On 02/11/2010 Anthony Liguori anth...@codemonkey.ws wrote: Oh, I see what's happening here. Yes, I think a leul_to_cpu() makes more sense. Maybe I'm missing something here. I couldn't find leul_to_cpu(), so have defined it in bswap.h. Correct? --- a/bswap.h +++ b/bswap.h @@ -205,8 +205,10 @@

Re: [PATCH 1/3] qemu-kvm: Wrap phys_ram_dirty with additional inline functions.

Why do you need a counter? It may be sufficient to set a single bit. This reduces the memory overhead and perhaps cache thrashing. Thanks for looking into this. I agree with your opinion. Our motivation here is to skip traveling when the dirty bitmap is really sparse or dense, so either

Re: [PATCH] qemu-kvm: Speed up of the dirty-bitmap-traveling

Please reuse the changelog when reposing a patch, this makes it easier for me to apply it. Thanks. Will follow it from next time. Should be a host long size, not guest. This will fail when running a 32-bit qemu-system-x86_64 binary. Sorry. That was our mistake. Instead of using a

[PATCH v2] qemu-kvm: Speed up of the dirty-bitmap-traveling

dirty-bitmap-traveling is carried out by byte size in qemu-kvm.c. But We think that dirty-bitmap-traveling by long size is faster than by byte size especially when most of memory is not dirty. Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- bswap.h|1 - qemu-kvm.c | 30

Re: [PATCH] qemu-kvm: Speed up of the dirty-bitmap-traveling

Thank you for your comments. We have implemented the code which applied your comments. This is patch for qemu-kvm.c. Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- qemu-kvm.c | 31 +++ 1 files changed, 19 insertions(+), 12 deletions(-) diff --git a/qemu

Re: [PATCH] qemu-kvm: Speed up of the dirty-bitmap-traveling

Thank you for your comments. We have implemented the code which applied your comments. This is patch for upstream. Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- kvm-all.c | 54 +++--- 1 files changed, 39 insertions(+), 15 deletions

[PATCH 3/3] qemu-kvm: Change the methods of get dirty pages.

Get number of the dirty and non-dirty pages using cpu_physical_memory_get_{dirty|non_dirty}_range(). Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- vl.c | 57 ++--- 1 files changed, 38 insertions(+), 19 deletions(-) diff --git

[PATCH 2/3] Check continuous dirty and non-dirty pages.

We will check the dirty and non-dirty pages as follows: 1. Checked by 64 pages block. Check pages using phys_ram_dirty_by_word. Count up/down it when phys_ram_dirty when is 0xff or not. 2. Checked by TARGET_LONG_SIZE pages block. 3. Checked by one page. Signed-off-by: OHMURA Kei ohmura

[PATCH 0/3] qemu-kvm: Check the dirty and non-dirty pages by multiple size

The dirty and non-dirty pages are checked one by one in vl.c. Since we believe that most of the memory is not dirty, checking the dirty and non-dirty pages by multiple page size should be much faster than checking them one by one. We think there is mostly two kind of situation. One is almost all

[PATCH 1/3] qemu-kvm: Wrap phys_ram_dirty with additional inline functions.

-dirty using phys_ram_dirty_by_word. Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- cpu-all.h | 74 cpu-defs.h |1 + 2 files changed, 75 insertions(+), 0 deletions(-) diff --git a/cpu-all.h b/cpu-all.h index 8ed76c7

Re: [PATCH] qemu-kvm: Speed up of the dirty-bitmap-traveling

Would be great if you could provide a version for upstream as well because it will likely replace this qemu-kvm code on day. O.K. We'll prepare it. We have implemented the version for upstream. Some source code are borrowed from qemu-kvm.c. It is not fully tested yet, though. We also did

Re: [PATCH] qemu-kvm: Speed up of the dirty-bitmap-traveling

Sounds logical - do you have numbers on the improvement? Sure. The patch showed approximately 3-7 times speed up when measured with rdtsc. The test environment and detailed results are described below. --- tmp = rdtsc(); /* function of original code*/ t1 += rdtsc() - tmp; tmp = rdtsc();

[PATCH] qemu-kvm: Speed up of the dirty-bitmap-traveling

dirty-bitmap-traveling is carried out by byte size in qemu-kvm.c. But We think that dirty-bitmap-traveling by long size is faster than by byte size especially when most of memory is not dirty. Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp --- qemu-kvm.c | 49