Re: [Qemu-devel] [PATCH v6 07/11] migration: poll the cm event while wait RDMA work request completion

[Juan Quintela]

Lidong Chen  wrote:
> From: Lidong Chen 
>
> If the peer qemu is crashed, the qemu_rdma_wait_comp_channel function
> maybe loop forever. so we should also poll the cm event fd, and when
> receive RDMA_CM_EVENT_DISCONNECTED and RDMA_CM_EVENT_DEVICE_REMOVAL,
> we consider some error happened.
>
> Signed-off-by: Lidong Chen 
> Signed-off-by: Gal Shachaf 
> Signed-off-by: Aviad Yehezkel 

Reviewed-by: Juan Quintela 

Re: [Qemu-devel] [PATCH v6 07/11] migration: poll the cm event while wait RDMA work request completion

[Dr. David Alan Gilbert]

* Lidong Chen (jemmy858...@gmail.com) wrote:
> From: Lidong Chen 
> 
> If the peer qemu is crashed, the qemu_rdma_wait_comp_channel function
> maybe loop forever. so we should also poll the cm event fd, and when
> receive RDMA_CM_EVENT_DISCONNECTED and RDMA_CM_EVENT_DEVICE_REMOVAL,
> we consider some error happened.
> 
> Signed-off-by: Lidong Chen 
> Signed-off-by: Gal Shachaf 
> Signed-off-by: Aviad Yehezkel 

I found the doc in the man page for rdma_create_event_channel
that said 'Users may make the fd non-blocking, poll or select the fd,
etc', so:


Reviewed-by: Dr. David Alan Gilbert 

> ---
>  migration/rdma.c | 33 ++---
>  1 file changed, 30 insertions(+), 3 deletions(-)
> 
> diff --git a/migration/rdma.c b/migration/rdma.c
> index d6bbf28..673f126 100644
> --- a/migration/rdma.c
> +++ b/migration/rdma.c
> @@ -1489,6 +1489,9 @@ static uint64_t qemu_rdma_poll(RDMAContext *rdma, 
> uint64_t *wr_id_out,
>   */
>  static int qemu_rdma_wait_comp_channel(RDMAContext *rdma)
>  {
> +struct rdma_cm_event *cm_event;
> +int ret = -1;
> +
>  /*
>   * Coroutine doesn't start until migration_fd_process_incoming()
>   * so don't yield unless we know we're running inside of a coroutine.
> @@ -1505,13 +1508,37 @@ static int qemu_rdma_wait_comp_channel(RDMAContext 
> *rdma)
>   * without hanging forever.
>   */
>  while (!rdma->error_state  && !rdma->received_error) {
> -GPollFD pfds[1];
> +GPollFD pfds[2];
>  pfds[0].fd = rdma->comp_channel->fd;
>  pfds[0].events = G_IO_IN | G_IO_HUP | G_IO_ERR;
> +pfds[0].revents = 0;
> +
> +pfds[1].fd = rdma->channel->fd;
> +pfds[1].events = G_IO_IN | G_IO_HUP | G_IO_ERR;
> +pfds[1].revents = 0;
> +
>  /* 0.1s timeout, should be fine for a 'cancel' */
> -switch (qemu_poll_ns(pfds, 1, 100 * 1000 * 1000)) {
> +switch (qemu_poll_ns(pfds, 2, 100 * 1000 * 1000)) {
> +case 2:
>  case 1: /* fd active */
> -return 0;
> +if (pfds[0].revents) {
> +return 0;
> +}
> +
> +if (pfds[1].revents) {
> +ret = rdma_get_cm_event(rdma->channel, _event);
> +if (!ret) {
> +rdma_ack_cm_event(cm_event);
> +}
> +
> +error_report("receive cm event while wait comp channel,"
> + "cm event is %d", cm_event->event);
> +if (cm_event->event == RDMA_CM_EVENT_DISCONNECTED ||
> +cm_event->event == RDMA_CM_EVENT_DEVICE_REMOVAL) {
> +return -EPIPE;
> +}
> +}
> +break;
>  
>  case 0: /* Timeout, go around again */
>  break;
> -- 
> 1.8.3.1
> 
--
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

[Qemu-devel] [PATCH v6 07/11] migration: poll the cm event while wait RDMA work request completion

[Lidong Chen]

From: Lidong Chen 

If the peer qemu is crashed, the qemu_rdma_wait_comp_channel function
maybe loop forever. so we should also poll the cm event fd, and when
receive RDMA_CM_EVENT_DISCONNECTED and RDMA_CM_EVENT_DEVICE_REMOVAL,
we consider some error happened.

Signed-off-by: Lidong Chen 
Signed-off-by: Gal Shachaf 
Signed-off-by: Aviad Yehezkel 
---
 migration/rdma.c | 33 ++---
 1 file changed, 30 insertions(+), 3 deletions(-)

diff --git a/migration/rdma.c b/migration/rdma.c
index d6bbf28..673f126 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -1489,6 +1489,9 @@ static uint64_t qemu_rdma_poll(RDMAContext *rdma, 
uint64_t *wr_id_out,
  */
 static int qemu_rdma_wait_comp_channel(RDMAContext *rdma)
 {
+struct rdma_cm_event *cm_event;
+int ret = -1;
+
 /*
  * Coroutine doesn't start until migration_fd_process_incoming()
  * so don't yield unless we know we're running inside of a coroutine.
@@ -1505,13 +1508,37 @@ static int qemu_rdma_wait_comp_channel(RDMAContext 
*rdma)
  * without hanging forever.
  */
 while (!rdma->error_state  && !rdma->received_error) {
-GPollFD pfds[1];
+GPollFD pfds[2];
 pfds[0].fd = rdma->comp_channel->fd;
 pfds[0].events = G_IO_IN | G_IO_HUP | G_IO_ERR;
+pfds[0].revents = 0;
+
+pfds[1].fd = rdma->channel->fd;
+pfds[1].events = G_IO_IN | G_IO_HUP | G_IO_ERR;
+pfds[1].revents = 0;
+
 /* 0.1s timeout, should be fine for a 'cancel' */
-switch (qemu_poll_ns(pfds, 1, 100 * 1000 * 1000)) {
+switch (qemu_poll_ns(pfds, 2, 100 * 1000 * 1000)) {
+case 2:
 case 1: /* fd active */
-return 0;
+if (pfds[0].revents) {
+return 0;
+}
+
+if (pfds[1].revents) {
+ret = rdma_get_cm_event(rdma->channel, _event);
+if (!ret) {
+rdma_ack_cm_event(cm_event);
+}
+
+error_report("receive cm event while wait comp channel,"
+ "cm event is %d", cm_event->event);
+if (cm_event->event == RDMA_CM_EVENT_DISCONNECTED ||
+cm_event->event == RDMA_CM_EVENT_DEVICE_REMOVAL) {
+return -EPIPE;
+}
+}
+break;
 
 case 0: /* Timeout, go around again */
 break;
-- 
1.8.3.1