[slurm-dev] mpi ranks not properly distributed

Hi Slurm Devs, I'm using 17.02.2 with MVAPICH2-GDR 2.2-5. I'm having a couple of problems, seemingly both Slurm and MPI related. But first, the problem I see immediately with my Slurm installation is that it is not distributing MPI processes correctly. When I run a simple test with 'srun

[slurm-dev] Re: Announcing Slurm Job Packs

Apologies, I was given incorrect information. The correct URLs are as you indicate: https://github.com/RJMS-Bull/slurm/tree/slurm-17.02-jobpacks and https://github.com/RJMS-Bull/slurm/blob/slurm-17.02-jobpacks/doc/html/job_packs.shtml -Original Message- From: Benjamin Redling

[slurm-dev] Re: slurm + openmpi + suspend problem

Good to hear! I think it’ll be out pretty soon (i.e., next few weeks) > On Jul 19, 2017, at 11:22 AM, Eugene Dedits wrote: > > Ralph, > > it seems to work now. Thanks a bunch! > When do you think we should expect 3.0.0 release? > > Best, > Eugene. > > > > > On

[slurm-dev] Re: slurm + openmpi + suspend problem

Ralph, it seems to work now. Thanks a bunch! When do you think we should expect 3.0.0 release? Best, Eugene. On Tue, Jul 18, 2017 at 1:07 PM, r...@open-mpi.org wrote: > Okay, I tracked it down and have a fix pending for OMPI master: >

[slurm-dev] RE: [Non-DoD Source] Re: General Post-Processing Question (UNCLASSIFIED)

CLASSIFICATION: UNCLASSIFIED Thanks Pete. That looks exactly like what I need. I couldn't think of the right search term, but pipeline is exactly what I'm trying to do. Thanks! Tony -Original Message- From: Peter A Ruprecht [mailto:peter.rupre...@colorado.edu] Sent: Wednesday, July

[slurm-dev] Remote license SLURM

Hello all. I was looking the Licenses Guide of Slurm (https://slurm.schedmd.com/licenses.html) and I was wondering if someone can explain me better how really works the remote license feature in Slurm... * Is it possible to force users to specify the "-L" flag (license type and number

[slurm-dev] Re: General Post-Processing Question (UNCLASSIFIED)

Tony, Have you considered using Slurm job dependencies for this workflow? That way you can submit the initial job and the post-processing job at the same time, but set a dependency on the post-processing job so that it can't start until the first job has finished successfully. We've had

[slurm-dev] General Post-Processing Question (UNCLASSIFIED)

CLASSIFICATION: UNCLASSIFIED Got a general question, but one that might be specifically addressed by Slurm - don't know. We have a multi-process, distributed simulation that runs as a single job and generates a significant amount of data. At the end of that run, we would like to be able to

[slurm-dev] Re: Dynamic, small partition?

You could try a QoS with Flags=DenyOnLimit,OverPartQOS,PartitionTimeLimit Priority= Depending on how you have your accounting set up, you could tweak some of the GrpTRES, MaxTRES, MaxTRESPU and MaxJobsPU to try to limit resource usage down to your 20 node limit. I'm not sure, off the top of my

[slurm-dev] Dynamic, small partition?

Is it possible to define, and use, a subset of all available nodes in a partition *without* explicitly setting aside a number of nodes (in a static Nodes=... definition)? Let's say I want, starting with a 100-node cluster, make 20 nodes available for jobs needing an extended MaxTime and Priority

[slurm-dev] Banking and Proposal System

Hello All, Disclaimer: I have no idea if anyone woud be interested in this. I needed this tool, wrote some code, and figured I should share it with the community. Recently, our group at Pitt moved from PBS to Slurm, but we didn't have an equivalent for Gold. We wanted to have a Gold-like bank,

[slurm-dev] Re: Change in srun ?

Title: Re: [slurm-dev] Re: Change in srun ? Take a look at https://www.cyberciti.biz/faq/rebuilding-ubuntu-debian-linux-binary-package/ Make sure slurm is already installed before rebuilding the package, and pass the correct configure flags (--with-pmi=

[slurm-dev] Re: Change in srun ?

Title: Re: [slurm-dev] Re: Change in srun ? Did you rebuild mpi with flag "--with-pmi" pointing at slurm's include dir, after making sure slurm has pmi.h there? I usually build with both --with-pmi and --with-slurm, although the last one should be enabled by default.

[slurm-dev] Re: Change in srun ?

Il 19/07/2017 09:37, Gilles Gouaillardet ha scritto: > this is probably due to a change in the PMI interface. Glip! > i suggest you rebuilt your MPI library first, and then try again That's problematic... I can only install from deb files, hoping package maintainers know what they're doing

[slurm-dev] Re: Change in srun ?

Diego, this is probably due to a change in the PMI interface. i suggest you rebuilt your MPI library first, and then try again Cheers, Gilles On 7/19/2017 4:20 PM, Diego Zuccato wrote: Hello all. I've just upgraded from Debian 8 to Debian 9, and that upgraded slurm from 14.03 to

[slurm-dev] Change in srun ?

Hello all. I've just upgraded from Debian 8 to Debian 9, and that upgraded slurm from 14.03 to 16.05. But now some MPI jobs are no more really parallel if run via srun, but work OK if run via mpirun. The problem seems that srun launches N threads with mpi_world_size=1 (so every process thinks