> Yes, the delay seems to be coming here:
>
> err = hpmp_rdmacm->rdma_connect(id, NULL);
> if (err) {
> hpmp_printf("rdma_connect() failed");
> return NULL;
> }
> t1 = MPI_Wtime();
>
> retry3:
> err =
> hpmp_rdmacm->rdma_get_cm_event(hpmp_rdmacm->connect_cm_channel, &event);
> if (err) {
> if (errno == EINTR) goto retry3;
> hpmp_printf("rdma_get_cm_event() failed");
> return NULL;
> }
>
> if (event->event != RDMA_CM_EVENT_ESTABLISHED) {
> hpmp_printf("rdma_get_cm_event() unexpected event (%d vs
> %d)"
> "while connecting to %d\n",
> event->event, RDMA_CM_EVENT_ESTABLISHED,
> port);
> return NULL;
> }
>
> t2 = MPI_Wtime();
> fprintf(stderr, "CONNECTION ESTABLISHED ON CONNECT %lf\n", t2-t1);
> hpmp_rdmacm->rdma_ack_cm_event(event);
>
>
>
> I get output such as:
>
> [ 1] CONNECTION ESTABLISHED ON CONNECT 0.001447
> [ 9] CONNECTION ESTABLISHED ON CONNECT 6.145778
> [ 6] CONNECTION ESTABLISHED ON CONNECT 5.233660
> [ 0] CONNECTION ESTABLISHED ON CONNECT 0.001343
> [ 6] CONNECTION ESTABLISHED ON CONNECT 0.001155
> [ 7] CONNECTION ESTABLISHED ON CONNECT 4.517944
> [11] CONNECTION ESTABLISHED ON CONNECT 0.001445
> [ 3] CONNECTION ESTABLISHED ON CONNECT 0.001558
> [ 7] CONNECTION ESTABLISHED ON CONNECT 0.001627
> [ 5] CONNECTION ESTABLISHED ON CONNECT 6.145470
> [ 2] CONNECTION ESTABLISHED ON CONNECT 5.657639
> [ 9] CONNECTION ESTABLISHED ON CONNECT 0.001602
> [10] CONNECTION ESTABLISHED ON CONNECT 6.188743
> [ 1] CONNECTION ESTABLISHED ON CONNECT 0.001500
> [ 6] CONNECTION ESTABLISHED ON CONNECT 0.001061
> [ 1] CONNECTION ESTABLISHED ON CONNECT 0.001183
> [11] CONNECTION ESTABLISHED ON CONNECT 0.001213
> [ 5] CONNECTION ESTABLISHED ON CONNECT 0.210666
What version of Linux is this running? Is the remote side responding to the
connection request in a timely manner? The connect call cannot complete until
the remote side accepts the connection.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
