[Qemu-devel] [Qemu-devel PATCH v3 1/2] sd: Fix out-of-bounds assertions

2019-06-19 Thread Lidong Chen
Due to an off-by-one error, the assert statements allow an out-of-bound array access. Signed-off-by: Lidong Chen Reviewed-by: Liam Merwick Reviewed-by: Marc-André Lureau Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Li Qiang Reviewed-by: Darren Kenny --- hw/sd/sd.c | 4 ++-- 1 file

[Qemu-devel] [Qemu-devel PATCH v3 2/2] util/main-loop: Fix incorrect assertion

2019-06-19 Thread Lidong Chen
The check for poll_fds in g_assert() was incorrect. The correct assertion should check "n_poll_fds + w->num <= ARRAY_SIZE(poll_fds)" because the subsequent for-loop is doing access to poll_fds[n_poll_fds + i] where i is in [0, w->num). Signed-off-by: Lidong Chen Suggeste

[Qemu-devel] [Qemu-devel PATCH v3 0/2] fix incorrect assertions in sd and main-loop

2019-06-19 Thread Lidong Chen
. Fix to silent static analyzer warnings." As confirmed by Lidong in v1 here: https://lists.gnu.org/archive/html/qemu-devel/2019-04/msg01337.html Thanks, Phil. --- Lidong Chen (2): sd: Fix out-of-bounds assertions util/main-loop: Fix incorrect assertion hw/sd/sd.c | 4 ++--

Re: [Qemu-devel] [Qemu-devel PATCH v2 2/2] util/main-loop: Fix incorrect assertion

2019-06-07 Thread Lidong Chen
Hi Philippe, On 6/6/2019 4:18 PM, Philippe Mathieu-Daudé wrote: On 6/5/19 9:15 PM, Lidong Chen wrote: The check for poll_fds in g_assert() was incorrect. The correct assertion should check "n_poll_fds + w->num <= ARRAY_SIZE(poll_fds)" because the subsequent for-loop is doing ac

[Qemu-devel] [Qemu-devel PATCH v2 2/2] util/main-loop: Fix incorrect assertion

2019-06-05 Thread Lidong Chen
The check for poll_fds in g_assert() was incorrect. The correct assertion should check "n_poll_fds + w->num <= ARRAY_SIZE(poll_fds)" because the subsequent for-loop is doing access to poll_fds[n_poll_fds + i] where i is in [0, w->num). Signed-off-by: Lidong Chen Suggeste

[Qemu-devel] [Qemu-devel PATCH v2 0/2] Fix incorrect assertions in sd and main-loop

2019-06-05 Thread Lidong Chen
v1: Patch1 fixed out-of-bounds assertions in hw/sd/sd.c v2: Added patch2 that fixed the incorrect assertion for poll_fds in util/main-loop.c based on the feedbacks from Peter Maydell and Liam Merwick, Lidong Chen (2): sd: Fix out-of-bounds assertions util/main-loop: Fix incorrect

[Qemu-devel] [Qemu-devel PATCH v2 1/2] sd: Fix out-of-bounds assertions

2019-06-05 Thread Lidong Chen
Due to an off-by-one error, the assert statements allow an out-of-bound array access. Signed-off-by: Lidong Chen Reviewed-by: Liam Merwick Reviewed-by: Marc-André Lureau Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Li Qiang Reviewed-by: Darren Kenny --- hw/sd/sd.c | 4 ++-- 1 file

Re: [Qemu-devel] [Qemu-devel PATCH v2 2/2] util/main-loop: Fix incorrect assertion

2019-06-05 Thread Lidong Chen
On 6/5/2019 1:57 AM, Philippe Mathieu-Daudé wrote: This patch doesn't seem related to the series cover. I will resent the patch to include more details to the cover. On 6/5/19 8:21 AM, Lidong Chen wrote: The check for poll_fds in g_assert() was incorrect. The correct assertion should check

[Qemu-devel] [Qemu-devel PATCH v2 2/2] util/main-loop: Fix incorrect assertion

2019-06-05 Thread Lidong Chen
The check for poll_fds in g_assert() was incorrect. The correct assertion should check "n_poll_fds + w->num <= ARRAY_SIZE(poll_fds)" because the subsequent for-loop is doing access to poll_fds[n_poll_fds + i] where i is in [0, w->num). Signed-off-by: Lidong Chen Reviewed-by: L

[Qemu-devel] [Qemu-devel PATCH v2 0/2] sd: Fix out-of-bounds assertions

2019-06-05 Thread Lidong Chen
The v2 changes include the fix for the incorrect assertion for poll_fds in util/main-loop.c according to the review comments from the previous sd fixes. Lidong Chen (2): sd: Fix out-of-bounds assertions util/main-loop: Fix incorrect assertion hw/sd/sd.c | 4 ++-- util/main-loop.c | 2

[Qemu-devel] [Qemu-devel PATCH v2 1/2] sd: Fix out-of-bounds assertions

2019-06-05 Thread Lidong Chen
Due to an off-by-one error, the assert statements allow an out-of-bound array access. Signed-off-by: Lidong Chen Reviewed-by: Liam Merwick Reviewed-by: Marc-André Lureau Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Li Qiang Reviewed-by: Darren Kenny --- hw/sd/sd.c | 4 ++-- 1 file

Re: [Qemu-devel] [PATCH] sd: Fix out-of-bounds assertions

2019-04-10 Thread Lidong Chen
, Markus Armbruster wrote: Lidong Chen writes: Due to an off-by-one error, the assert statements allow an out-of-bounds array access. Signed-off-by: Lidong Chen Reviewed-by: Liam Merwick ---   hw/sd/sd.c | 4 ++--   1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/hw/sd/sd.c b/hw

Re: [Qemu-devel] [PATCH] sd: Fix out-of-bounds assertions

2019-04-08 Thread Lidong Chen
Hi Philippe, On 4/8/2019 2:27 PM, Philippe Mathieu-Daudé wrote: Hi Lidong On 4/8/19 9:04 PM, Lidong Chen wrote: Due to an off-by-one error, the assert statements allow an out-of-bounds array access. ... which can't happen. Thus harmless for 4.0. I suppose this is a static analysis warning

[Qemu-devel] [PATCH] sd: Fix out-of-bounds assertions

2019-04-08 Thread Lidong Chen
Due to an off-by-one error, the assert statements allow an out-of-bounds array access. Signed-off-by: Lidong Chen --- hw/sd/sd.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/hw/sd/sd.c b/hw/sd/sd.c index aaab15f..818f86c 100644 --- a/hw/sd/sd.c +++ b/hw/sd/sd.c

[Qemu-devel] [PATCH v6 09/11] migration: poll the cm event for destination qemu

2018-08-06 Thread Lidong Chen
The destination qemu only poll the comp_channel->fd in qemu_rdma_wait_comp_channel. But when source qemu disconnnect the rdma connection, the destination qemu should be notified. Signed-off-by: Lidong Chen --- migration/migration.c | 3 ++- migration/rdma.c |

[Qemu-devel] [PATCH v6 11/11] migration: create a dedicated thread to release rdma resource

2018-08-06 Thread Lidong Chen
this will cause the guest os hang for a while when migration finished. So create a dedicated thread to release rdma resource. Signed-off-by: Lidong Chen --- migration/migration.c | 6 ++ migration/migration.h | 3 +++ migration/rdma.c | 47

[Qemu-devel] [PATCH v6 07/11] migration: poll the cm event while wait RDMA work request completion

2018-08-06 Thread Lidong Chen
From: Lidong Chen If the peer qemu is crashed, the qemu_rdma_wait_comp_channel function maybe loop forever. so we should also poll the cm event fd, and when receive RDMA_CM_EVENT_DISCONNECTED and RDMA_CM_EVENT_DEVICE_REMOVAL, we consider some error happened. Signed-off-by: Lidong Chen Signed

[Qemu-devel] [PATCH v6 05/11] migration: implement io_set_aio_fd_handler function for RDMA QIOChannel

2018-08-06 Thread Lidong Chen
From: Lidong Chen if qio_channel_rdma_readv return QIO_CHANNEL_ERR_BLOCK, the destination qemu crash. The backtrace is: (gdb) bt #0 0x in ?? () #1 0x008db50e in qio_channel_set_aio_fd_handler (ioc=0x38111e0, ctx=0x3726080, io_read=0x8db841 , io_write

[Qemu-devel] [PATCH v6 03/11] migration: implement bi-directional RDMA QIOChannel

2018-08-06 Thread Lidong Chen
From: Lidong Chen This patch implements bi-directional RDMA QIOChannel. Because different threads may access RDMAQIOChannel currently, this patch use RCU to protect it. Signed-off-by: Lidong Chen Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c | 2 + migration/migration.c

[Qemu-devel] [PATCH v6 08/11] migration: implement the shutdown for RDMA QIOChannel

2018-08-06 Thread Lidong Chen
From: Lidong Chen Because RDMA QIOChannel not implement shutdown function, If the to_dst_file was set error, the return path thread will wait forever. and the migration thread will wait return path thread exit. the backtrace of return path thread is: (gdb) bt #0 0x7f372a76bb0f

[Qemu-devel] [PATCH v6 04/11] migration: Stop rdma yielding during incoming postcopy

2018-08-06 Thread Lidong Chen
From: Lidong Chen During incoming postcopy, the destination qemu will invoke qemu_rdma_wait_comp_channel in a seprate thread. So does not use rdma yield, and poll the completion channel fd instead. Signed-off-by: Lidong Chen Reviewed-by: Dr. David Alan Gilbert --- migration/rdma.c | 4

[Qemu-devel] [PATCH v6 01/11] migration: disable RDMA WRITE after postcopy started

2018-08-06 Thread Lidong Chen
From: Lidong Chen RDMA WRITE operations are performed with no notification to the destination qemu, then the destination qemu can not wakeup. This patch disable RDMA WRITE after postcopy started. Signed-off-by: Lidong Chen Reviewed-by: Dr. David Alan Gilbert --- migration/qemu-file.c | 8

[Qemu-devel] [PATCH v6 10/11] migration: remove the unnecessary RDMA_CONTROL_ERROR message

2018-08-06 Thread Lidong Chen
It's not necessary to send RDMA_CONTROL_ERROR when clean up rdma resource. If rdma->error_state is ture, the message may not send successfully. and the cm event can also notify the peer qemu. Signed-off-by: Lidong Chen --- migration/rdma.c | 11 --- 1 file changed, 11 deletions(-) d

[Qemu-devel] [PATCH v6 06/11] migration: invoke qio_channel_yield only when qemu_in_coroutine()

2018-08-06 Thread Lidong Chen
From: Lidong Chen when qio_channel_read return QIO_CHANNEL_ERR_BLOCK, the source qemu crash. The backtrace is: (gdb) bt #0 0x7fb20aba91d7 in raise () from /lib64/libc.so.6 #1 0x7fb20abaa8c8 in abort () from /lib64/libc.so.6 #2 0x7fb20aba2146 in __assert_fail_base

[Qemu-devel] [PATCH v6 02/11] migration: create a dedicated connection for rdma return path

2018-08-06 Thread Lidong Chen
From: Lidong Chen If start a RDMA migration with postcopy enabled, the source qemu establish a dedicated connection for return path. Signed-off-by: Lidong Chen Reviewed-by: Dr. David Alan Gilbert --- migration/rdma.c | 94 ++-- 1 file

[Qemu-devel] [PATCH v6 00/11] Enable postcopy RDMA live migration

2018-08-06 Thread Lidong Chen
close (Daniel) - destroy the mutex before free QEMUFile (David) - use rdmain and rmdaout instead of rdma->return_path (Daniel) [v2] - does not update bytes_xfer when disable RDMA WRITE (David) - implement bi-directional communication for RDMA QIOChannel (Daniel) Lidong Chen (11): migrat

[Qemu-devel] [PATCH v6 09/12] migration: implement the shutdown for RDMA QIOChannel

2018-08-03 Thread Lidong Chen
From: Lidong Chen Because RDMA QIOChannel not implement shutdown function, If the to_dst_file was set error, the return path thread will wait forever. and the migration thread will wait return path thread exit. the backtrace of return path thread is: (gdb) bt #0 0x7f372a76bb0f

[Qemu-devel] [PATCH v6 07/12] migration: invoke qio_channel_yield only when qemu_in_coroutine()

2018-08-03 Thread Lidong Chen
From: Lidong Chen when qio_channel_read return QIO_CHANNEL_ERR_BLOCK, the source qemu crash. The backtrace is: (gdb) bt #0 0x7fb20aba91d7 in raise () from /lib64/libc.so.6 #1 0x7fb20abaa8c8 in abort () from /lib64/libc.so.6 #2 0x7fb20aba2146 in __assert_fail_base

[Qemu-devel] [PATCH v6 03/12] migration: avoid concurrent invoke channel_close by different threads

2018-08-03 Thread Lidong Chen
From: Lidong Chen The channel_close maybe invoked by different threads. For example, source qemu invokes qemu_fclose in main thread, migration thread and return path thread. Destination qemu invokes qemu_fclose in main thread, listen thread and COLO incoming thread. Signed-off-by: Lidong Chen

[Qemu-devel] [PATCH v6 12/12] migration: create a dedicated thread to release rdma resource

2018-08-03 Thread Lidong Chen
this will cause the guest os hang for a while when migration finished. So create a dedicated thread to release rdma resource. Signed-off-by: Lidong Chen --- migration/migration.c | 6 ++ migration/migration.h | 3 +++ migration/rdma.c | 47

[Qemu-devel] [PATCH v6 11/12] migration: remove the unnecessary RDMA_CONTROL_ERROR message

2018-08-03 Thread Lidong Chen
It's not necessary to send RDMA_CONTROL_ERROR when clean up rdma resource. If rdma->error_state is ture, the message may not send successfully. and the cm event can also notify the peer qemu. Signed-off-by: Lidong Chen --- migration/rdma.c | 11 --- 1 file changed, 11 deletions(-) d

[Qemu-devel] [PATCH v6 02/12] migration: create a dedicated connection for rdma return path

2018-08-03 Thread Lidong Chen
From: Lidong Chen If start a RDMA migration with postcopy enabled, the source qemu establish a dedicated connection for return path. Signed-off-by: Lidong Chen Reviewed-by: Dr. David Alan Gilbert --- migration/rdma.c | 94 ++-- 1 file

[Qemu-devel] [PATCH v6 10/12] migration: poll the cm event for destination qemu

2018-08-03 Thread Lidong Chen
The destination qemu only poll the comp_channel->fd in qemu_rdma_wait_comp_channel. But when source qemu disconnnect the rdma connection, the destination qemu should be notified. Signed-off-by: Lidong Chen --- migration/migration.c | 3 ++- migration/rdma.c |

[Qemu-devel] [PATCH v6 08/12] migration: poll the cm event while wait RDMA work request completion

2018-08-03 Thread Lidong Chen
From: Lidong Chen If the peer qemu is crashed, the qemu_rdma_wait_comp_channel function maybe loop forever. so we should also poll the cm event fd, and when receive RDMA_CM_EVENT_DISCONNECTED and RDMA_CM_EVENT_DEVICE_REMOVAL, we consider some error happened. Signed-off-by: Lidong Chen Signed

[Qemu-devel] [PATCH v6 01/12] migration: disable RDMA WRITE after postcopy started

2018-08-03 Thread Lidong Chen
From: Lidong Chen RDMA WRITE operations are performed with no notification to the destination qemu, then the destination qemu can not wakeup. This patch disable RDMA WRITE after postcopy started. Signed-off-by: Lidong Chen Reviewed-by: Dr. David Alan Gilbert --- migration/qemu-file.c | 8

[Qemu-devel] [PATCH v6 04/12] migration: implement bi-directional RDMA QIOChannel

2018-08-03 Thread Lidong Chen
From: Lidong Chen This patch implements bi-directional RDMA QIOChannel. Because different threads may access RDMAQIOChannel currently, this patch use RCU to protect it. Signed-off-by: Lidong Chen Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c | 2 + migration/migration.c

[Qemu-devel] [PATCH v6 05/12] migration: Stop rdma yielding during incoming postcopy

2018-08-03 Thread Lidong Chen
From: Lidong Chen During incoming postcopy, the destination qemu will invoke qemu_rdma_wait_comp_channel in a seprate thread. So does not use rdma yield, and poll the completion channel fd instead. Signed-off-by: Lidong Chen Reviewed-by: Dr. David Alan Gilbert --- migration/rdma.c | 4

[Qemu-devel] [PATCH v6 06/12] migration: implement io_set_aio_fd_handler function for RDMA QIOChannel

2018-08-03 Thread Lidong Chen
From: Lidong Chen if qio_channel_rdma_readv return QIO_CHANNEL_ERR_BLOCK, the destination qemu crash. The backtrace is: (gdb) bt #0 0x in ?? () #1 0x008db50e in qio_channel_set_aio_fd_handler (ioc=0x38111e0, ctx=0x3726080, io_read=0x8db841 , io_write

[Qemu-devel] [PATCH v6 00/12] Enable postcopy RDMA live migration

2018-08-03 Thread Lidong Chen
close (Daniel) - destroy the mutex before free QEMUFile (David) - use rdmain and rmdaout instead of rdma->return_path (Daniel) [v2] - does not update bytes_xfer when disable RDMA WRITE (David) - implement bi-directional communication for RDMA QIOChannel (Daniel) Lidong Chen (12): migrat

[Qemu-devel] [PATCH 1/2] migration: fix the Unknown ending state error log

2018-07-24 Thread Lidong Chen
When cancelling migration, the state is MIGRATION_STATUS_CANCELLING. The state change to MIGRATION_STATUS_CANCELLED when cleanup_bh is scheduled. So when migration_iteration_finish is invoked, the state should be MIGRATION_STATUS_CANCELLING. Signed-off-by: Lidong Chen --- migration/migration.c

[Qemu-devel] [PATCH 2/2] migration: fix duplicate initialization for expected_downtime and cleanup_bh

2018-07-24 Thread Lidong Chen
migrate_fd_connect duplicate initialize expected_downtime and cleanup_bh. Signed-off-by: Lidong Chen --- migration/migration.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index ff05422..b8c4fec 100644 --- a/migration/migration.c +++ b

[Qemu-devel] [PATCH v2] migration: release MigrationIncomingState in migration_object_finalize

2018-07-18 Thread Lidong Chen
Qemu initialize the MigrationIncomingState structure in migration_object_init, but not release it. this patch release it in migration_object_finalize. Signed-off-by: Lidong Chen --- migration/migration.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/migration/migration.c b/migration

[Qemu-devel] [PATCH v5 10/10] migration: implement the shutdown for RDMA QIOChannel

2018-06-05 Thread Lidong Chen
837 #5 0x7f372aa49e25 in start_thread () from /lib64/libpthread.so.0 #6 0x7f372a77635d in clone () from /lib64/libc.so.6 Signed-off-by: Lidong Chen Reviewed-by: Dr. David Alan Gilbert --- migration/rdma.c | 40 1 file changed, 40 insertion

[Qemu-devel] [PATCH v5 06/10] migration: implement io_set_aio_fd_handler function for RDMA QIOChannel

2018-06-05 Thread Lidong Chen
From: Lidong Chen if qio_channel_rdma_readv return QIO_CHANNEL_ERR_BLOCK, the destination qemu crash. The backtrace is: (gdb) bt #0 0x in ?? () #1 0x008db50e in qio_channel_set_aio_fd_handler (ioc=0x38111e0, ctx=0x3726080, io_read=0x8db841 , io_write

[Qemu-devel] [PATCH v5 07/10] migration: invoke qio_channel_yield only when qemu_in_coroutine()

2018-06-05 Thread Lidong Chen
From: Lidong Chen when qio_channel_read return QIO_CHANNEL_ERR_BLOCK, the source qemu crash. The backtrace is: (gdb) bt #0 0x7fb20aba91d7 in raise () from /lib64/libc.so.6 #1 0x7fb20abaa8c8 in abort () from /lib64/libc.so.6 #2 0x7fb20aba2146 in __assert_fail_base

[Qemu-devel] [PATCH v5 04/10] migration: implement bi-directional RDMA QIOChannel

2018-06-05 Thread Lidong Chen
From: Lidong Chen This patch implements bi-directional RDMA QIOChannel. Because different threads may access RDMAQIOChannel currently, this patch use RCU to protect it. Signed-off-by: Lidong Chen --- migration/colo.c | 2 + migration/migration.c| 2 + migration/postcopy-ram.c

[Qemu-devel] [PATCH v5 09/10] migration: poll the cm event while wait RDMA work request completion

2018-06-05 Thread Lidong Chen
If the peer qemu is crashed, the qemu_rdma_wait_comp_channel function maybe loop forever. so we should also poll the cm event fd, and when receive RDMA_CM_EVENT_DISCONNECTED and RDMA_CM_EVENT_DEVICE_REMOVAL, we consider some error happened. Signed-off-by: Lidong Chen --- migration/rdma.c | 33

[Qemu-devel] [PATCH v5 03/10] migration: avoid concurrent invoke channel_close by different threads

2018-06-05 Thread Lidong Chen
The channel_close maybe invoked by different threads. For example, source qemu invokes qemu_fclose in main thread, migration thread and return path thread. Destination qemu invokes qemu_fclose in main thread, listen thread and COLO incoming thread. Signed-off-by: Lidong Chen --- migration

[Qemu-devel] [PATCH v5 05/10] migration: Stop rdma yielding during incoming postcopy

2018-06-05 Thread Lidong Chen
From: Lidong Chen During incoming postcopy, the destination qemu will invoke qemu_rdma_wait_comp_channel in a seprate thread. So does not use rdma yield, and poll the completion channel fd instead. Signed-off-by: Lidong Chen Reviewed-by: Dr. David Alan Gilbert --- migration/rdma.c | 4

[Qemu-devel] [PATCH v5 08/10] migration: create a dedicated thread to release rdma resource

2018-06-05 Thread Lidong Chen
this will cause the guest os hang for a while when migration finished. So create a dedicated thread to release rdma resource. Signed-off-by: Lidong Chen --- migration/rdma.c | 43 +++ 1 file changed, 27 insertions(+), 16 deletions(-) diff --git

[Qemu-devel] [PATCH v5 00/10] Enable postcopy RDMA live migration

2018-06-05 Thread Lidong Chen
DMA WRITE (David) - implement bi-directional communication for RDMA QIOChannel (Daniel) Lidong Chen (10): migration: disable RDMA WRITE after postcopy started migration: create a dedicated connection for rdma return path migration: avoid concurrent invoke channel_close by different thre

[Qemu-devel] [PATCH v5 02/10] migration: create a dedicated connection for rdma return path

2018-06-05 Thread Lidong Chen
From: Lidong Chen If start a RDMA migration with postcopy enabled, the source qemu establish a dedicated connection for return path. Signed-off-by: Lidong Chen Reviewed-by: Dr. David Alan Gilbert --- migration/rdma.c | 94 ++-- 1 file

[Qemu-devel] [PATCH v5 01/10] migration: disable RDMA WRITE after postcopy started

2018-06-05 Thread Lidong Chen
From: Lidong Chen RDMA WRITE operations are performed with no notification to the destination qemu, then the destination qemu can not wakeup. This patch disable RDMA WRITE after postcopy started. Signed-off-by: Lidong Chen Reviewed-by: Dr. David Alan Gilbert --- migration/qemu-file.c | 8

[Qemu-devel] [PATCH] migration: release MigrationIncomingState in migration_object_finalize

2018-06-04 Thread Lidong Chen
Qemu initialize the MigrationIncomingState structure in migration_object_init, but not release it. this patch release it in migration_object_finalize. Signed-off-by: Lidong Chen --- migration/migration.c | 7 +++ 1 file changed, 7 insertions(+) diff --git a/migration/migration.c b

[Qemu-devel] [PATCH v4 11/12] migration: poll the cm event while wait RDMA work request completion

2018-05-30 Thread Lidong Chen
If the peer qemu is crashed, the qemu_rdma_wait_comp_channel function maybe loop forever. so we should also poll the cm event fd, and when receive any cm event, we consider some error happened. Signed-off-by: Lidong Chen --- migration/rdma.c | 35 --- 1 file

[Qemu-devel] [PATCH v4 07/12] migration: not wait RDMA_CM_EVENT_DISCONNECTED event after rdma_disconnect

2018-05-30 Thread Lidong Chen
From: Lidong Chen When cancel migration during RDMA precopy, the source qemu main thread hangs sometime. The backtrace is: (gdb) bt #0 0x7f249eabd43d in write () from /lib64/libpthread.so.0 #1 0x7f24a1ce98e4 in rdma_get_cm_event (channel=0x4675d10, event=0x7ffe2f643dd0

[Qemu-devel] [PATCH v4 10/12] migration: create a dedicated thread to release rdma resource

2018-05-30 Thread Lidong Chen
this will cause the guest os hang for a while when migration finished. So create a dedicated thread to release rdma resource. Signed-off-by: Lidong Chen --- migration/rdma.c | 21 + 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/migration/rdma.c b/migration

[Qemu-devel] [PATCH v4 12/12] migration: implement the shutdown for RDMA QIOChannel

2018-05-30 Thread Lidong Chen
837 #5 0x7f372aa49e25 in start_thread () from /lib64/libpthread.so.0 #6 0x7f372a77635d in clone () from /lib64/libc.so.6 Signed-off-by: Lidong Chen --- migration/rdma.c | 40 1 file changed, 40 insertions(+) diff --git a/migration/rdma.c b

[Qemu-devel] [PATCH v4 05/12] migration: implement bi-directional RDMA QIOChannel

2018-05-30 Thread Lidong Chen
From: Lidong Chen This patch implements bi-directional RDMA QIOChannel. Because different threads may access RDMAQIOChannel currently, this patch use RCU to protect it. Signed-off-by: Lidong Chen --- migration/colo.c | 2 + migration/migration.c| 2 + migration/postcopy-ram.c

[Qemu-devel] [PATCH v4 06/12] migration: Stop rdma yielding during incoming postcopy

2018-05-30 Thread Lidong Chen
From: Lidong Chen During incoming postcopy, the destination qemu will invoke qemu_rdma_wait_comp_channel in a seprate thread. So does not use rdma yield, and poll the completion channel fd instead. Signed-off-by: Lidong Chen Reviewed-by: Dr. David Alan Gilbert --- migration/rdma.c | 4

[Qemu-devel] [PATCH v4 08/12] migration: implement io_set_aio_fd_handler function for RDMA QIOChannel

2018-05-30 Thread Lidong Chen
From: Lidong Chen if qio_channel_rdma_readv return QIO_CHANNEL_ERR_BLOCK, the destination qemu crash. The backtrace is: (gdb) bt #0 0x in ?? () #1 0x008db50e in qio_channel_set_aio_fd_handler (ioc=0x38111e0, ctx=0x3726080, io_read=0x8db841 , io_write

[Qemu-devel] [PATCH v4 04/12] migration: avoid concurrent invoke channel_close by different threads

2018-05-30 Thread Lidong Chen
From: Lidong Chen The channel_close maybe invoked by different threads. For example, source qemu invokes qemu_fclose in main thread, migration thread and return path thread. Destination qemu invokes qemu_fclose in main thread, listen thread and COLO incoming thread. Add a mutex in QEMUFile

[Qemu-devel] [PATCH v4 03/12] migration: remove unnecessary variables len in QIOChannelRDMA

2018-05-30 Thread Lidong Chen
From: Lidong Chen Because qio_channel_rdma_writev and qio_channel_rdma_readv maybe invoked by different threads concurrently, this patch removes unnecessary variables len in QIOChannelRDMA and use local variable instead. Signed-off-by: Lidong Chen Reviewed-by: Dr. David Alan Gilbert Reviewed

[Qemu-devel] [PATCH v4 01/12] migration: disable RDMA WRITE after postcopy started

2018-05-30 Thread Lidong Chen
From: Lidong Chen RDMA WRITE operations are performed with no notification to the destination qemu, then the destination qemu can not wakeup. This patch disable RDMA WRITE after postcopy started. Signed-off-by: Lidong Chen Reviewed-by: Dr. David Alan Gilbert --- migration/qemu-file.c | 8

[Qemu-devel] [PATCH v4 09/12] migration: invoke qio_channel_yield only when qemu_in_coroutine()

2018-05-30 Thread Lidong Chen
From: Lidong Chen when qio_channel_read return QIO_CHANNEL_ERR_BLOCK, the source qemu crash. The backtrace is: (gdb) bt #0 0x7fb20aba91d7 in raise () from /lib64/libc.so.6 #1 0x7fb20abaa8c8 in abort () from /lib64/libc.so.6 #2 0x7fb20aba2146 in __assert_fail_base

[Qemu-devel] [PATCH v4 00/12] Enable postcopy RDMA live migration

2018-05-30 Thread Lidong Chen
(David) - use rdmain and rmdaout instead of rdma->return_path (Daniel) [v2] - does not update bytes_xfer when disable RDMA WRITE (David) - implement bi-directional communication for RDMA QIOChannel (Daniel) Lidong Chen (12): migration: disable RDMA WRITE after postcopy started migration: cre

[Qemu-devel] [PATCH v4 02/12] migration: create a dedicated connection for rdma return path

2018-05-30 Thread Lidong Chen
From: Lidong Chen If start a RDMA migration with postcopy enabled, the source qemu establish a dedicated connection for return path. Signed-off-by: Lidong Chen Reviewed-by: Dr. David Alan Gilbert --- migration/rdma.c | 94 ++-- 1 file

[Qemu-devel] [PATCH 1/2] migration: implement io_set_aio_fd_handler function for RDMA QIOChannel

2018-05-07 Thread Lidong Chen
implement io_set_aio_fd_handler. so qio_channel_set_aio_fd_handler will access NULL pointer. Signed-off-by: Lidong Chen <lidongc...@tencent.com> --- migration/rdma.c | 16 1 file changed, 16 insertions(+) diff --git a/migration/rdma.c b/migration/rdma.c index 92e4d30

[Qemu-devel] [PATCH 2/2] migration: invoke qio_channel_yield only when qemu_in_coroutine()

2018-05-07 Thread Lidong Chen
osix.c:504 #12 0x7fb20af3ddc5 in start_thread () from /lib64/libpthread.so.0 #13 0x7fb20ac6b74d in clone () from /lib64/libc.so.6 This patch fixed by invoke qio_channel_yield only when qemu_in_coroutine(). Signed-off-by: Lidong Chen <lidongc...@tencent.com> --- migration/qem

[Qemu-devel] [PATCH 2/2] migration: not wait RDMA_CM_EVENT_DISCONNECTED event after rdma_disconnect

2018-05-06 Thread Lidong Chen
is also destroyed in qemu_rdma_cleanup. Signed-off-by: Lidong Chen <lidongc...@tencent.com> --- migration/rdma.c | 12 ++-- migration/trace-events | 1 - 2 files changed, 2 insertions(+), 11 deletions(-) diff --git a/migration/rdma.c b/migration/rdma.c index 0dd4033..92e4d30

[Qemu-devel] [PATCH 1/2] migration: update index field when delete or qsort RDMALocalBlock

2018-05-06 Thread Lidong Chen
. When vmpin is large than max locked memory limitation, ibv_reg_mr will failed, and migration can not start successfully again. Signed-off-by: Lidong Chen <lidongc...@tencent.com> --- migration/rdma.c | 7 +++ 1 file changed, 7 insertions(+) diff --git a/migration/rdma.c b/migration/

[Qemu-devel] [PATCH v3 6/6] migration: Stop rdma yielding during incoming postcopy

2018-05-05 Thread Lidong Chen
During incoming postcopy, the destination qemu will invoke qemu_rdma_wait_comp_channel in a seprate thread. So does not use rdma yield, and poll the completion channel fd instead. Signed-off-by: Lidong Chen <lidongc...@tencent.com> Reviewed-by: Dr. David Alan Gilbert <dgilb...@r

[Qemu-devel] [PATCH v3 4/6] migration: avoid concurrent invoke channel_close by different threads

2018-05-05 Thread Lidong Chen
concurrent invoke channel_close. Signed-off-by: Lidong Chen <lidongc...@tencent.com> --- migration/qemu-file.c | 5 + 1 file changed, 5 insertions(+) diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 977b9ae..87d0f05 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-

[Qemu-devel] [PATCH v3 2/6] migration: create a dedicated connection for rdma return path

2018-05-05 Thread Lidong Chen
If start a RDMA migration with postcopy enabled, the source qemu establish a dedicated connection for return path. Signed-off-by: Lidong Chen <lidongc...@tencent.com> Reviewed-by: Dr. David Alan Gilbert <dgilb...@redhat.com> --- migration

[Qemu-devel] [PATCH v3 5/6] migration: implement bi-directional RDMA QIOChannel

2018-05-05 Thread Lidong Chen
This patch implements bi-directional RDMA QIOChannel. Because different threads may access RDMAQIOChannel currently, this patch use RCU to protect it. Signed-off-by: Lidong Chen <lidongc...@tencent.com> --- migration/colo.c | 2 + migration/migration.c| 2 + migration/po

[Qemu-devel] [PATCH v3 3/6] migration: remove unnecessary variables len in QIOChannelRDMA

2018-05-05 Thread Lidong Chen
Because qio_channel_rdma_writev and qio_channel_rdma_readv maybe invoked by different threads concurrently, this patch removes unnecessary variables len in QIOChannelRDMA and use local variable instead. Signed-off-by: Lidong Chen <lidongc...@tencent.com> Reviewed-by: Dr. David Alan Gilbert

[Qemu-devel] [PATCH v3 0/6] Enable postcopy RDMA live migration

2018-05-05 Thread Lidong Chen
ate bytes_xfer when disable RDMA WRITE (David) - implement bi-directional communication for RDMA QIOChannel (Daniel) Lidong Chen (6): migration: disable RDMA WRITE after postcopy started migration: create a dedicated connection for rdma return path migration: remove unnecessary variables

[Qemu-devel] [PATCH v3 1/6] migration: disable RDMA WRITE after postcopy started

2018-05-05 Thread Lidong Chen
RDMA WRITE operations are performed with no notification to the destination qemu, then the destination qemu can not wakeup. This patch disable RDMA WRITE after postcopy started. Signed-off-by: Lidong Chen <lidongc...@tencent.com> Reviewed-by: Dr. David Alan Gilbert <dgilb...@r

[Qemu-devel] [PATCH v2 2/5] migration: create a dedicated connection for rdma return path

2018-04-25 Thread Lidong Chen
If start a RDMA migration with postcopy enabled, the source qemu establish a dedicated connection for return path. Signed-off-by: Lidong Chen <lidongc...@tencent.com> Signed-off-by: Dr. David Alan Gilbert <dgilb...@redhat.com> --- migration

[Qemu-devel] [PATCH v2 1/5] migration: disable RDMA WRITE after postcopy started

2018-04-25 Thread Lidong Chen
RDMA WRITE operations are performed with no notification to the destination qemu, then the destination qemu can not wakeup. This patch disable RDMA WRITE after postcopy started. Signed-off-by: Lidong Chen <lidongc...@tencent.com> --- migration/qemu-file.c | 8 ++-- migration/

[Qemu-devel] [PATCH v2 5/5] migration: Stop rdma yielding during incoming postcopy

2018-04-25 Thread Lidong Chen
During incoming postcopy, the destination qemu will invoke qemu_rdma_wait_comp_channel in a seprate thread. So does not use rdma yield, and poll the completion channel fd instead. Signed-off-by: Lidong Chen <lidongc...@tencent.com> --- migration/rdma.c | 4 +++- 1 file changed, 3 inse

[Qemu-devel] [PATCH v2 3/5] migration: remove unnecessary variables len in QIOChannelRDMA

2018-04-25 Thread Lidong Chen
Because qio_channel_rdma_writev and qio_channel_rdma_readv maybe invoked by different threads concurrently, this patch removes unnecessary variables len in QIOChannelRDMA and use local variable instead. Signed-off-by: Lidong Chen <lidongc...@tencent.com> --- migration/rdma.

[Qemu-devel] [PATCH v2 0/5] Enable postcopy RDMA live migration

2018-04-25 Thread Lidong Chen
. This patch just make postcopy works, and will improve performance later. [v2] - does not update bytes_xfer when disable RDMA WRITE (David) - implement bi-directional communication for RDMA QIOChannel (Daniel) Please, review. Lidong Chen (5): migration: disable RDMA WRITE after postcopy

[Qemu-devel] [PATCH v2 4/5] migration: implement bi-directional RDMA QIOChannel

2018-04-25 Thread Lidong Chen
This patch implements bi-directional RDMA QIOChannel. Because different threads may access RDMAQIOChannel concurrently, this patch use RCU to protect it. Signed-off-by: Lidong Chen <lidongc...@tencent.com> --- migration/rdma.c | 162 +---

[Qemu-devel] [PATCH 5/5] migration: disable RDMA WRITR after postcopy started.

2018-04-07 Thread Lidong Chen
RDMA write operations are performed with no notification to the destination qemu, then the destination qemu can not wakeup. So disable RDMA WRITE after postcopy started. Signed-off-by: Lidong Chen <lidongc...@tencent.com> --- migration/qemu-file.c | 3 ++- migration/rdma.c

[Qemu-devel] [PATCH 4/5] migration: fix qemu carsh when RDMA live migration

2018-04-07 Thread Lidong Chen
After postcopy, the destination qemu work in the dedicated thread, so only invoke yield_until_fd_readable before postcopy migration. Signed-off-by: Lidong Chen <lidongc...@tencent.com> --- migration/rdma.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/migration/rd

[Qemu-devel] [PATCH 2/5] migration: add the interface to set get_return_path

2018-04-07 Thread Lidong Chen
The default get_return_path function of iochannel does not work for RDMA live migration. So add the interface to set get_return_path. Signed-off-by: Lidong Chen <lidongc...@tencent.com> --- migration/qemu-file-channel.c | 12 migration/qemu-file.c | 10 -- mig

[Qemu-devel] [PATCH 1/5] migration: create a dedicated connection for rdma return path

2018-04-07 Thread Lidong Chen
If start a RDMA migration with postcopy enabled, the source qemu establish a dedicated connection for return path. Signed-off-by: Lidong Chen <lidongc...@tencent.com> --- migration/rdma.c | 94 ++-- 1 file changed, 91 insertions(+), 3 del

[Qemu-devel] [PATCH 0/5] Enable postcopy RDMA live migration

2018-04-07 Thread Lidong Chen
Current Qemu RDMA communication does not support send and receive data at the same time, so when RDMA live migration with postcopy enabled, the source qemu return path thread get qemu file error. Those patch add the postcopy support for RDMA live migration. Lidong Chen (5): migration: create

[Qemu-devel] [PATCH 3/5] migration: implement the get_return_path for RDMA iochannel

2018-04-07 Thread Lidong Chen
the default get_return_path function does not work for RDMA live migration, the patch implement the get_return_path for RDMA iochannel. Signed-off-by: Lidong Chen <lidongc...@tencent.com> --- migration/rdma.c | 38 ++ 1 file changed, 38 insertions(+)

[Qemu-devel] [PATCH] migration: Fix rate limiting issue on RDMA migration

2018-03-10 Thread Lidong Chen
RDMA migration implement save_page function for QEMUFile, but ram_control_save_page do not increase bytes_xfer. So when doing RDMA migration, it will use whole bandwidth. Signed-off-by: Lidong Chen <lidongc...@tencent.com> --- migration/qemu-file.c | 2 +- 1 file changed, 1 insertion

[Qemu-devel] [Bug 1687578] [NEW] when migrate vm, reboot in guest os, the guest os sometime hang

2017-05-02 Thread Lidong Chen
Public bug reported: qemu version:v2.9.0-rc5 release 1.virsh migrate --live 165cf436-312f-47e7-90f2-f8aa63f34893 --copy-storage-inc qemu+ssh://10.59.163.38/system 2.run reboot in guest os, add reboot in /etc/rc.local 3.guest os hang sometime. strace output of qemu: ppoll([{fd=9,

[Qemu-devel] [Bug 1687569] [NEW] when migration cancel, qemu main thread hung

2017-05-02 Thread Lidong Chen
Public bug reported: qemu version:v2.9.0-rc5 release 1.virsh migrate --live 165cf436-312f-47e7-90f2-f8aa63f34893 --copy-storage-all qemu+ssh://10.59.163.38/system 2.press Ctrl+C cancel migrate qemu main thread hung (gdb) bt #0 0x7fca9f4574b7 in ppoll () from /lib64/libc.so.6 #1

[Qemu-devel] [Bug 1681688] Re: qemu live migration failed

2017-04-11 Thread Lidong Chen
blk->root->perm is 1 when blk_new_open. the blk->root->perm is update to 3 during virtio_blk_device_realize. but after this commit, the blk->root->perm is still 1. and cause bdrv_aligned_pwritev failed. Breakpoint 1, blk_set_perm (blk=0x14c32b0, perm=3, shared_perm=29, errp=0x7fffd380) at

[Qemu-devel] [Bug 1681688] [NEW] qemu live migration failed

2017-04-11 Thread Lidong Chen
Public bug reported: qemu live migration failed the dest qemu report this error. Receiving block device images Completed 0 %^Mqemu-system-x86_64: block/io.c:1348: bdrv_aligned_pwritev: Assertion `child->perm & BLK_PERM_WRITE' failed. this bug is caused by this patch:

[Qemu-devel] [RFC] migration/block:limit the time used for block migration

2017-03-28 Thread Lidong Chen
when migration with quick speed, mig_save_device_bulk invoke bdrv_is_allocated too frequently, and cause vnc reponse slowly. this patch limit the time used for bdrv_is_allocated. Signed-off-by: Lidong Chen <lidongc...@tencent.com> --- migration/block.

[Qemu-devel] [PATCH v2] migration/block: Avoid invoking blk_drain too frequently

2017-03-14 Thread Lidong Chen
ove the migration speed from 505856 rsec/s to 855756 rsec/s. Signed-off-by: Lidong Chen <jemmy858...@gmail.com> --- migration/block.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/migration/block.c b/migration/block.c index 6741228..7734ff7 100644 --- a/migration/block.c +++ b/migratio

[Qemu-devel] [Bug 1469924] [NEW] qemu-kvm crash when guest os is booting

2015-06-29 Thread Lidong Chen
Public bug reported: this is the command line of qemu. 2015-06-30 01:52:59.508+: starting up LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=none /usr/bin/qemu-kvm -name rhel7 -S -machine pc-i440fx-2.1,accel=kvm,usb=off -cpu SandyBridge -m 2048 -realtime