Hi, all, I'm very glad to inform you that CHPL_GASNET_SEGMENT=large works for my benchmark! Now I can run the benchmark without any error. Thank you all so much!
> Here's another tip from the GASNet team. Would you be able to give this a > try and see if it helps with the ibv + fast? Thanks for your information. Maybe I can try the tip next week. Best, Akihiro On Jan 31, 2014, at 12:15 PM, Brad Chamberlain wrote: > > Hi Rafael -- > > Here's another tip from the GASNet team. Would you be able to give this a > try and see if it helps with the ibv + fast? > > ---- > > The Bad Address sounds like an interaction between IBV and POSIX Shared > Memory: > https://upc-bugs.lbl.gov/bugzilla/show_bug.cgi?id=2929 > > The work-around is to configure GASNet with "--enable-pshm-sysv > --disable-pshm-posix". > > ---- > > Thanks, > -Brad > > > On Thu, 30 Jan 2014, Brad Chamberlain wrote: > >> >> Hi Rafael -- >> >> Thanks for your analysis and description. >> >> I don't have any problem with changing the default segment for 'ibv' and >> agree that it sounds like the right thing to do if it works around this >> issue. It seems like 'fast' would be preferable to 'large' from a >> performance perspective (based on a quick glance at the docs -- I'm mostly >> unfamiliar with 'large'). >> >> In trying to use the 'fast' segment, do you know whether the memory >> allocator used changed to either tcmalloc or dlmalloc (either automatically >> or manually)? That is probably necessary for correctness, but should've >> (hopefully) happened automatically. printchplenv can be used to verify. >> >> Note that I believe you should only need to set CHPL_GASNET_SEGMENT. The >> CHPL_MAKE_ variables should follow automatically (and are not intended to be >> set by the end-user). >> >> Thanks again, >> -Brad >> >> >> On Thu, 30 Jan 2014, Rafael Larrosa Jiménez wrote: >> >>> El Jueves, 30 de enero de 2014 12:21:11 Akihiro Hayashi escribió: >>>> Hi, >>>> Thanks for your helping me to understand the problem. >>>> I think now I understand the problem correctly. >>>> Rafael Asenjo and Rafael Larrosa. >>>> I'm pretty sure my assignment have involved a contiguous block and >>>> useBulkTransfer is active and useBulkTransferStride is not active in my >>>> chapel compiler. I just wanted to make both sure my code (useBulkTransfer) >>>> and Rafaerl Larrosa's code (useBulkTransferStride) can fail due to the >>>> ibv-conduit bug (https://upc-bugs.lbl.gov/bugzilla/show_bug.cgi?id=495). >>> First, tell that I'm not the developer, but a client of that code. >>> Second, I have been writting this email for several hours, as I changed my >>> vision on the problem, have been reading the ib verbs manuals, ib code, etc. >>> At the end I have found the solution, by default chapel puts all memory as >>> memory accesible by RDMA, which seems to be the problem, as it can be >>> "unpinned, if instead only "large" portions are used for RDMA then it works >>> fine, as explained in one of the messages: >>> --- >>> In a GASNET_SEGMENT_FAST or _LARGE configuration the segment is obtained at >>> startup via mmap() and is never unmapped. So, sending from inside the >>> GASNet >>> segment in these two cases will ensure this bug cannot occur. >>> --- >>> So if you define : >>> export CHPL_MAKE_GASNET_SEGMENT=large >>> export CHPL_MAKE_COMM_SEGMENT=large >>> export CHPL_GASNET_SEGMENT=large >>> Then do a make clobber followed by a make and recompile your program, it >>> should work fine. >>> BTW, when using portals, gemini or aries, the fast segment is used, as >>> explained in : >>> doc/release/README.multilocale >>> --- >>> 3) Advanced GASNet users can set CHPL_GASNET_SEGMENT to choose a >>> memory segment to use with GASNet. Current defaults are: >>> >>> When CHPL_COMM_SUBSTRATE is... Chapel will choose... >>> portals fast >>> gemini fast >>> (other) everything >>> --- >>> But using fast with ibv gives this error: >>> *** FATAL ERROR: Unexpected error Bad address (rc=1 errno=14) when >>> registering >>> the segment >>> But that is not a problem as large segments are the solution, hope they >>> don't >>> break other things :-) >>> I have tested it with Akihiro code, and it seems to work fine now. >>> Perphaps is a good idea to change the default to use large segments with >>> ibv. >>> Greets, >>> Rafael >>> -- >>> Rafael Larrosa Jiménez >>> Centro de Supercomputación y Bioinformática - http://www.scbi.uma.es >>> Universidad de Málaga >>> EMAIL: [email protected] Edificio de Bioinnovación >>> TELEF: + 34951952788 C/ Severo Ochoa 34 >>> FAX : +34951952792 Parque Tecnológico de Andalucía >>> 29590 Málaga (SPAIN) ------------------------------------------------------------------------------ WatchGuard Dimension instantly turns raw network data into actionable security intelligence. It gives you real-time visual feedback on key security issues and trends. Skip the complicated setup - simply import a virtual appliance and go from zero to informed in seconds. http://pubads.g.doubleclick.net/gampad/clk?id=123612991&iu=/4140/ostg.clktrk _______________________________________________ Chapel-developers mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/chapel-developers
