El Jueves, 30 de enero de 2014 12:21:11 Akihiro Hayashi escribió:
> Hi,
>
> Thanks for your helping me to understand the problem.
> I think now I understand the problem correctly.
>
> Rafael Asenjo and Rafael Larrosa.
> I'm pretty sure my assignment have involved a contiguous block and
> useBulkTransfer is active and useBulkTransferStride is not active in my
> chapel compiler. I just wanted to make both sure my code (useBulkTransfer)
> and Rafaerl Larrosa's code (useBulkTransferStride) can fail due to the
> ibv-conduit bug (https://upc-bugs.lbl.gov/bugzilla/show_bug.cgi?id=495).
First, tell that I'm not the developer, but a client of that code.
Second, I have been writting this email for several hours, as I changed my
vision on the problem, have been reading the ib verbs manuals, ib code, etc.
At the end I have found the solution, by default chapel puts all memory as
memory accesible by RDMA, which seems to be the problem, as it can be
"unpinned, if instead only "large" portions are used for RDMA then it works
fine, as explained in one of the messages:
---
In a GASNET_SEGMENT_FAST or _LARGE configuration the segment is obtained at
startup via mmap() and is never unmapped. So, sending from inside the GASNet
segment in these two cases will ensure this bug cannot occur.
---
So if you define :
export CHPL_MAKE_GASNET_SEGMENT=large
export CHPL_MAKE_COMM_SEGMENT=large
export CHPL_GASNET_SEGMENT=large
Then do a make clobber followed by a make and recompile your program, it
should work fine.
BTW, when using portals, gemini or aries, the fast segment is used, as
explained in :
doc/release/README.multilocale
---
3) Advanced GASNet users can set CHPL_GASNET_SEGMENT to choose a
memory segment to use with GASNet. Current defaults are:
When CHPL_COMM_SUBSTRATE is... Chapel will choose...
portals fast
gemini fast
(other) everything
---
But using fast with ibv gives this error:
*** FATAL ERROR: Unexpected error Bad address (rc=1 errno=14) when registering
the segment
But that is not a problem as large segments are the solution, hope they don't
break other things :-)
I have tested it with Akihiro code, and it seems to work fine now.
Perphaps is a good idea to change the default to use large segments with ibv.
Greets,
Rafael
--
Rafael Larrosa Jiménez
Centro de Supercomputación y Bioinformática - http://www.scbi.uma.es
Universidad de Málaga
EMAIL: [email protected] Edificio de Bioinnovación
TELEF: + 34951952788 C/ Severo Ochoa 34
FAX : +34951952792 Parque Tecnológico de Andalucía
29590 Málaga (SPAIN)
------------------------------------------------------------------------------
WatchGuard Dimension instantly turns raw network data into actionable
security intelligence. It gives you real-time visual feedback on key
security issues and trends. Skip the complicated setup - simply import
a virtual appliance and go from zero to informed in seconds.
http://pubads.g.doubleclick.net/gampad/clk?id=123612991&iu=/4140/ostg.clktrk
_______________________________________________
Chapel-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/chapel-developers
