For LCRC, it is better to follow the example on their website (http://www.lcrc.anl.gov/for-users/using-lcrc/running-jobs/running-jobs-on-bebop/)
#!/bin/bash #SBATCH -J mympijobname #SBATCH -A myaccount #SBATCH -p knlall #SBATCH -C knl,quad,cache #SBATCH -N 2 #SBATCH --ntasks-per-node=64 #SBATCH -t 00:15:00 export I_MPI_FABRICS=shm:tmi srun ./ex_modify I would not recommend you to use flat mode unless you really want to seriously optimize your code for best performance on KNL. When configuring petsc on KNL, adding options --with-avx512-kernels --with-memalign=64 may speed up your code a bit. Hong (Mr.) On Feb 27, 2019, at 5:02 PM, Sajid Ali via petsc-users <[email protected]<mailto:[email protected]>> wrote: Hi Junchao, I’m confused with the syntax. If I submit the following as my job script, I get an error : #!/bin/bash #SBATCH --job-name=petsc_test #SBATCH -N 1 #SBATCH -C knl,quad,flat #SBATCH -p apsxrmd #SBATCH --time=1:00:00 module load intel/18.0.3-d6gtsxs module load intel-parallel-studio/cluster.2018.3-xvnfrfz module load numactl-2.0.12-intel-18.0.3-wh44iog srun -n 64 -c 64 --cpu_bind=cores numactl -m 1 aps ./ex_modify -ts_type cn -prop_steps 25 -pc_type gamg -ts_monitor -log_view The error is : srun: cluster configuration lacks support for cpu binding srun: error: Unable to create step for job 916208: More processors requested than permitted I’m following the advice as given at slide 33 of https://www.nersc.gov/assets/Uploads/02-using-cori-knl-nodes-20170609.pdf For further info, I’m using LCRC at ANL. Thank You, Sajid Ali Applied Physics Northwestern University
