Our cluster compute nodes are running an older OS (Centos 6.6) with old out-of-the-box libraries (gcc 4.4.7). On our login node we have newer compilers installed so that we can use contemporary C/C++ language features. The problem is that our chapel executables sometimes use features that are only to be found in newer libraries (ex. libstdc++, etc...) built with the newer compilers so the executables (*_real) need to find the newer libraries when they get to the compute nodes. (The libraries _are_ installed on the compute nodes, but not in the usual /usr/lib* locations so they won't be found by default.
Normally SLURM will propagate the environment (esp. LD_LIBRARY_PATH) so things get found with the appropriate environment set up, but with chapel (under slurm-gasnetrun_ibv) SLURM is only used to allocate nodes, but gasnetrun_ibv actually executes the node (*_real) code and doesn't seem to propagate environment by default. There is a -E option to gasnetrun_ibv.pl and if I hack the expect script to add -E LD_LIBRARY_PATH things work, but I don't see a way to get chapel to generate it's wrappers to slip in appropriate -E's.
I hacked gasnetrun_ibv.pl to propagate LD_LIBRARY_PATH by default and things work, but it seems like there should be a cleaner solution. I'm thinking I'm not the first to run into this so I was
wondering if anybody's get a better way to deal with it. Thanks. _______________________________________________ Chapel-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/chapel-users
