Re: [OMPI users] iWARP usage issue

2016-03-08 Thread Nathan Hjelm
See https://github.com/open-mpi/ompi/pull/1439 I was seeing this problem when enabling CUDA support as it sets btl_openib_max_send_size to 128k but does not change the receive queue settings. Tested the commit in #1439 and it fixes the issue for me. -Nathan On Tue, Mar 08, 2016 at 03:57:39PM +0

Re: [OMPI users] iWARP usage issue

2016-03-08 Thread Nathan Hjelm
This is a bug we need to deal with. If we are getting queue pair settings from an ini file and the max_send_size if the default value we should set the max send size to the size of the largest queue pair. I will work on a fix. -Nathan On Tue, Mar 08, 2016 at 03:57:39PM +0900, Gilles Gouaillardet

Re: [OMPI users] iWARP usage issue

2016-03-08 Thread Gilles Gouaillardet
Per the error message, can you try to mpirun --mca btl_openib_if_include cxgb3_0 --mca btl_openib_max_send_size 65536 ... and see whether it helps ? you can also try various settings for the receive queue, for example edit your /.../share/openmpi/mca-btl-openib-device-params.ini and set the