WIEN2k has a usersguide, where the different parallelization modes are extensively described.

On a cluster with a queuing system (like SLURM) it should not even be possible to access nodes (except the frontend) via ssh without using SLURM (on our SLURM machine ssh is possible only to nodes which are assigned to a user by salloc or a sbatch job), thus overloading can be prevented.

We ALWAYS run our jobs using SLURM and typically submit them using

sbatch slurm.job

Now you mentioned correctly, that wien2k needs a ".machine" file and thus "slurm.job" has to create it on the fly.

I've provided an example script (which you may need to adapt for user or resource specifications) at


One more hint: in the file $WIENROOT/parallel_options one specifies if k-point parallel (USE_REMOTE) and mpi-parallel (MPI_REMOTE) jobs are started using ssh (1) or not (0).

setenv USE_REMOTE 1 or 0
setenv MPI_REMOTE 0

On modern mpi-versions use always MPI_REMOTE=0.
Usually k-point parallelism is meaningful only for up to 8 (then set OMP_NUM_THREAD=2) or 16 cores, otherwise the overhead is too big. In such cases one would use only ONE node and it runs as a "shared-memory machine (USE_REMOTE=0) (without mpi at all). On medium sized cases with a few k-points and larger matrices, a "mixed" k-parallel and mpi-parallel setup is best and is used in the slurm.job example above.

PS: I'm sending this also to our WIEN2k-mailing list, because this is of general interest and I don't want to write the same email all the time again. PPS: Please use in general the mailing list (see www.wien2k.at), as I normally do not answer questions directly sent to me.

Best regards
Peter Blaha

On 11/11/2015 07:08 PM, Robb III, George B. wrote:
Hi Dr. Schwartz / Dr. Blaha-

We have noticed on our SLURM <http://schedmd.com/#index> based research
cluster that WIEN2k suite commands take advantage of a .machines file to
spawn off ssh sessions to individual nodes vs using a scheduler.

We have SLURM configured to control cluster resource allocations and
have collisions of resources when ssh processes are called from the
WIEN2k suite.

e.g. SLURM controls node2 and has 95% allocated resources for jobs, but
WIEN2k process is launched from the head node it will ssh to node2 (due
to the 5% free resources) and spawn additional un-SLURM-managed
processes on node2.  Node2 is now over subscribed.

Does the WIEN2k suite have an administrative guide or documentation?
Does the WIEN2k suite have recommend cluster manager (i.e. PBS, SLURM,
LSF, etc..)?

Thanks again for any assistance and looking forward having the labs on
our campus use the WIEN2k suite.


George B. Robb III
Systems Administrator
Research Computing Support Services - (RCSS)
University of Missouri System


Peter BLAHA, Inst.f. Materials Chemistry, TU Vienna, A-1060 Vienna
Phone: +43-1-58801-165300             FAX: +43-1-58801-165982
Email: bl...@theochem.tuwien.ac.at    WIEN2k: http://www.wien2k.at
WWW:   http://www.imc.tuwien.ac.at/staff/tc_group_e.php
Wien mailing list

Reply via email to