Should be fixed with r32058
On Jun 20, 2014, at 4:13 AM, tmish...@jcity.maeda.co.jp wrote: > > > Hi Ralph, > > By the way, something is wrong with your latest rmaps_rank_file.c. > I've got the error below. I'm tring to find the problem. But, you > could find it more quickly... > > [mishima@manage trial]$ cat rankfile > rank 0=node05 slot=0-1 > rank 1=node05 slot=3-4 > rank 2=node05 slot=6-7 > [mishima@manage trial]$ mpirun -np 3 -rf rankfile -report-bindings > demos/myprog > -------------------------------------------------------------------------- > Error, invalid syntax in the rankfile (rankfile) > syntax must be the fallowing > rank i=host_i slot=string > Examples of proper syntax include: > rank 1=host1 slot=1:0,1 > rank 0=host2 slot=0:* > rank 2=host4 slot=1-2 > rank 3=host3 slot=0:1;1:0-2 > -------------------------------------------------------------------------- > [manage.cluster:24456] [[20979,0],0] ORTE_ERROR_LOG: Bad parameter in file > rmaps_rank_file.c at line 483 > [manage.cluster:24456] [[20979,0],0] ORTE_ERROR_LOG: Bad parameter in file > rmaps_rank_file.c at line 149 > [manage.cluster:24456] [[20979,0],0] ORTE_ERROR_LOG: Bad parameter in file > base/rmaps_base_map_job.c at line 287 > > Regards, > Tetsuya Mishima > >> My guess is that the coll/ml component may have problems with binding a > single process across multiple cores like that - it might be that we'll > have to have it check for that condition and disqualify >> itself. It is a particularly bad binding pattern, though, as shared > memory gets completely messed up when you split that way. >> >> >> On Jun 19, 2014, at 3:57 PM, tmish...@jcity.maeda.co.jp wrote: >> >>> >>> Hi folks, >>> >>> Recently I have been seeing a hang with trunk when I specify a >>> particular binding by use of rankfile or "-map-by slot". >>> >>> This can be reproduced by the rankfile which allocates a process >>> beyond socket boundary. For example, on the node05 which has 2 socket >>> with 4 core, the rank 1 is allocated through socket 0 and 1 as shown >>> below. Then it hangs in the middle of communication. >>> >>> [mishima@manage trial]$ cat rankfile1 >>> rank 0=node05 slot=0-1 >>> rank 1=node05 slot=3-4 >>> rank 2=node05 slot=6-7 >>> >>> [mishima@manage trial]$ mpirun -rf rankfile1 -report-bindings > demos/myprog >>> [node05.cluster:02342] MCW rank 0 bound to socket 0[core 0[hwt 0]], > socket >>> 0[core 1[hwt 0]]: [B/B/./.][./././.] >>> [node05.cluster:02342] MCW rank 1 bound to socket 0[core 3[hwt 0]], > socket >>> 1[core 4[hwt 0]]: [./././B][B/././.] >>> [node05.cluster:02342] MCW rank 2 bound to socket 1[core 6[hwt 0]], > socket >>> 1[core 7[hwt 0]]: [./././.][././B/B] >>> Hello world from process 2 of 3 >>> Hello world from process 1 of 3 >>> << hang here! >> >>> >>> If I disable coll_ml or use 1.8 series, it works, which means it >>> might be affected by coll_ml component, I guess. But, unfortunately, >>> I have no idea to fix this problem. So, please somebody could resolve >>> the issue. >>> >>> [mishima@manage trial]$ mpirun -rf rankfile1 -report-bindings -mca >>> coll_ml_priority 0 demos/myprog >>> [node05.cluster:02382] MCW rank 0 bound to socket 0[core 0[hwt 0]], > socket >>> 0[core 1[hwt 0]]: [B/B/./.][./././.] >>> [node05.cluster:02382] MCW rank 1 bound to socket 0[core 3[hwt 0]], > socket >>> 1[core 4[hwt 0]]: [./././B][B/././.] >>> [node05.cluster:02382] MCW rank 2 bound to socket 1[core 6[hwt 0]], > socket >>> 1[core 7[hwt 0]]: [./././.][././B/B] >>> Hello world from process 2 of 3 >>> Hello world from process 0 of 3 >>> Hello world from process 1 of 3 >>> >>> In addtition, when I use the host with 12 cores, "-map-by slot" causes > the >>> same problem. >>> [mishima@manage trial]$ mpirun -np 3 -map-by slot:pe=4 -report-bindings >>> demos/myprog >>> [manage.cluster:02557] MCW rank 0 bound to socket 0[core 0[hwt 0]], > socket >>> 0[core 1[hwt 0]], socket 0[core 2[hwt 0]], so >>> cket 0[core 3[hwt 0]]: [B/B/B/B/./.][./././././.] >>> [manage.cluster:02557] MCW rank 1 bound to socket 0[core 4[hwt 0]], > socket >>> 0[core 5[hwt 0]], socket 1[core 6[hwt 0]], so >>> cket 1[core 7[hwt 0]]: [././././B/B][B/B/./././.] >>> [manage.cluster:02557] MCW rank 2 bound to socket 1[core 8[hwt 0]], > socket >>> 1[core 9[hwt 0]], socket 1[core 10[hwt 0]], s >>> ocket 1[core 11[hwt 0]]: [./././././.][././B/B/B/B] >>> Hello world from process 1 of 3 >>> Hello world from process 2 of 3 >>> << hang here! >> >>> >>> Regards, >>> Tetsuya Mishima >>> >>> _______________________________________________ >>> devel mailing list >>> de...@open-mpi.org >>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel >>> Link to this post: > http://www.open-mpi.org/community/lists/devel/2014/06/15030.php >> >> _______________________________________________ >> devel mailing list >> de...@open-mpi.org >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel >> Link to this post: > http://www.open-mpi.org/community/lists/devel/2014/06/15032.php > > _______________________________________________ > devel mailing list > de...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel > Link to this post: > http://www.open-mpi.org/community/lists/devel/2014/06/15039.php
