On Tuesday 06 May 2008 09:41:53 am Jeff Squyres wrote: > I actually don't know what the RDMA CM requires for the LMC>0 case -- > does it require a unique IP address for every LID?
It requires a unique IP address for every hca/port in use by rdmacm. I see the bug in rdmacm (since I don't believe you were trying to use rdmacm), and will have a patch out shortly. > > On May 6, 2008, at 5:09 AM, Lenny Verkhovsky wrote: > > > Hi, > > > > running BW benchmark with btl_openib_max_lmc >= 2 couses warning > > ( MPI from the TRUNK ) !!!! > > > > > > #mpirun --bynode -np 40 -hostfile hostfile_ompi_arbel -mca > > btl_openib_max_lmc 2 ./mpi_p_LMC -t bw -s 400000 > > BW (40) (size min max avg) 400000 321.493757 > > 342.972837 329.493715 > > > > #mpirun --bynode -np 40 -hostfile hostfile_ompi_arbel -mca > > btl_openib_max_lmc 3 ./mpi_p_LMC -t bw -s 400000 > > [witch9][[7493,1],7][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch2][[7493,1],0][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch10][[7493,1],9][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch6][[7493,1],4][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch4][[7493,1],2][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch7][[7493,1],5][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch2][[7493,1],10][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch9][[7493,1],17][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch5][[7493,1],3][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch8][[7493,1],6][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch6][[7493,1],14][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch10][[7493,1],19][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch5][[7493,1],13][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch4][[7493,1],12][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch9][[7493,1],27][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch5][[7493,1],23][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch2][[7493,1],20][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch9][[7493,1],37][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch7][[7493,1],35][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch4][[7493,1],32][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch4][[7493,1],22][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch5][[7493,1],33][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch2][[7493,1],30][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch8][[7493,1],16][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch7][[7493,1],15][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch10][[7493,1],39][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch7][[7493,1],25][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch10][[7493,1],29][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch6][[7493,1],34][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch8][[7493,1],26][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch6][[7493,1],24][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > [witch8][[7493,1],36][../../../../../ompi/mca/btl/openib/connect/ > > btl_openib_connect_rdmacm.c:989:create_message] No IP address found > > BW (40) (size min max avg) 400000 312.622582 > > 334.037277 324.014814 > > > > using -mca btl openib,self causes warning with LMC >=10 > > > > > > Best regards > > Lenny. > > _______________________________________________ > > devel mailing list > > de...@open-mpi.org > > http://www.open-mpi.org/mailman/listinfo.cgi/devel > >
