Elliot,
The top of your head must be a nice place because that worked. Just to
prove to myself, I added the `here` stuff to the bottom of my code, results
before and after setting HFI_NO_CPUAFFINITY=1.
Pi is approximately: 3.14159
1
1
28
export HFI_NO_CPUAFFINITY=1
Pi is approximately: 3.14159
28
28
28
I will put some documentation together with my pull request for running on
OmniPath w/ Slurm.
- Barry
On Tue, Mar 28, 2017 at 2:08 AM, Elliot Ronaghan <[email protected]> wrote:
> Hi Barry,
>
> Off the top of my head I'd recommend turning off any external CPU
> affinity binding. Try doing:
> `export HFI_NO_CPUAFFINITY=1`
>
> We had another user run into a similar issue where PSM2 was limiting
> the chapel executable to only have affinity to 1 processor:
> https://github.com/chapel-lang/chapel/issues/5536#issuecomment-285826038
>
> If that doesn't help, can you send us the slurm command you're using as
> well as the output of `$CHPL_HOME/util/printchplenv --anonymize` and
> `printenv | grep -i affinity`?
>
> It'd also help to see how much parallelism and how many physical cores a
> chapel program thinks it has access too by running this program:
>
> writeln(here.maxTaskPar); // per-locale parallelism
> writeln(here.numPUs(accessible=true)); // # cores we have access to
> writeln(here.numPUs(accessible=false)); // # cores on the locale
>
> Elliot
>
>
> -----Original Message-----
> From: Barry Moore <[email protected]>
> Date: Monday, March 27, 2017 at 8:06 PM
> To: "[email protected]" <chapel-users@lists.
> sourceforge.net>
> Subject: Looking for guidance with Chapel+Slurm+OmniPath
>
> Hello All,
>
>
> Looking to get my feet off the ground with Chapel on our HPC resources
> here at Pitt. I wrote the following approximation to Pi code:
>
>
> ```
> // Need BlockDist module to use Block distribution mapping
> use BlockDist;
>
>
> // Some config constants for running the problem
> config const tasksPerLocale : int = 1;
> config const numberGridPoints : int = 10;
>
>
> // The Grid and Domain specification
> const Grid = {1..numberGridPoints};
> const Domain: domain(1) dmapped Block(boundingBox = Grid) = Grid;
>
>
> // Our approximation procedure
> proc approx(a: real) return 4.0 / (1.0 + a ** 2);
>
>
> // The separation of grid points and the sum which will catch a reduce
> var dGrid : real = 1.0 / numberGridPoints;
> var sum : real = 0.0;
>
>
> // forall values in the domain calculation the valueOnGrid and approx of
> that value
> forall d in Domain with (+ reduce sum) {
> var valueOnGrid : real = dGrid * (d + 0.5);
> sum += approx(valueOnGrid);
> }
>
>
> // Print out Pi
> writeln("Pi is approximately: ", sum * dGrid);
> ```
>
>
> using the pull request: https://github.com/chapel-lang/chapel/pull/5843
> to compile the code.
>
>
> I am convinced the code is running over OmniPath, but I am quite sure it
> is not using multiple cores per locale. Am I missing an additional level in
> my forall loop (something with tasksPerLocale)?
>
>
> Any help is appreciated.
>
>
> Thanks,
>
>
> Barry
>
>
> --
> Barry E Moore II, PhD
> E-mail: [email protected]
>
> Assistant Research Professor
> Center for Simulation and Modeling
> University of Pittsburgh
> Pittsburgh, PA 15260
>
>
>
>
>
>
>
>
>
>
>
--
Barry E Moore II, PhD
E-mail: [email protected]
Assistant Research Professor
Center for Simulation and Modeling
University of Pittsburgh
Pittsburgh, PA 15260
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Chapel-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/chapel-users
