Yes it is wrong.
a) The memory count usually as memory/core, so if you have a 64GB node,
you should most likely divide by the number of cores (and then maybe
multiply by 2, because you can use multithreading with 2 cores and use 2
cores/job.
b) The memory is calculated as: NMATMAX**2 * 10 (actually it should be
8 bytes/number, but we need also some overhead)
10000**2*10=10**9 bytes = 1GB
NMATMAX=20000: ~4GB (per core !!)
c) larger values than 20000 are most likely useless, because as was said
by L.Marks, for larger cases there is no k-parallism but even when it
would fit in memory, it would run much too long. Therefore we switch to
mpi-parallelization with many cores (on a good cluster AND big problem
we have used up to 1000 cores).
And for these problems NMTMAX will be internally increased by
NMATMAX*sqrt(NP)
On 11/04/2015 03:04 PM, tongwei wrote:
Thanks.
For the NMATMAX and NUME values, I am following the prompt during
installation:
"NMATMAX=10000 ==> 1GB (real) (==> cells with about 50-150 atoms/unitcell)
==> for lapw1c: NMATMAX will be reduced internally to NMATMAX/sqrt2
==> for lapw1_mpi: NMATMAX will be increased internally to
NMATMAX*sqrt(NP)
NUME determines the number of states to output. As a rule of thumb one can
estimate 100 basis functions per atom in the cell and 10 occupied states
per
atom, so set NUME=NMATMAX/10.
"
I have 128GB at node1 and 64GB memory at node2 . So, I assume that 32GB
should be available for WIEN2k. Then 32*10000=320000,
NUME=NMATMAX/10=32000.
Is this wrong? Then, where to change this value?
Best,
WT
------------------------------------------------------------------------
Date: Mon, 2 Nov 2015 09:01:41 -0600
From: l-ma...@northwestern.edu
To: wien@zeus.theochem.tuwien.ac.at
Subject: Re: [Wien] Some error messages during Installation of
WIEN2k_14.2 on CentOS6.5
I am 99.9% certain that none of these is an error. The compile script
will tell you if there are any at the end, these are flagged with "Error".
N.B., the values
set value for NMATMAX=320000
set value for NUME=32000
are large, you could run into problems with these if you are running
k-point parallel for larger jobs. Since you only have a 2 node cluster
with 1G ethernet (?) you may be only intending to run smaller jobs in
which case it will be fine. You could get an OOM (Out Of Memory) OS
crash with larger jobs.
On Mon, Nov 2, 2015 at 8:51 AM, tongwei <weitong...@hotmail.com
<mailto:weitong...@hotmail.com>> wrote:
Dear Everyone,
During installing WIEN2k_14.2, I watched the screen and saw a few
errors in the following. I am not sure it is OK or not.
-------after save compiler options-------------------
No Makefile.orig in SRC_structeditor/SRC_lib, leaving directory.
No Makefile.orig in SRC_BerryPI, leaving directory.
No Makefile.orig in SRC_lib, leaving directory.
No Makefile.orig in SRC_structeditor, leaving directory.
No Makefile.orig in SRC_templates, leaving directory.
No Makefile.orig in SRC_usersguide_html, leaving directory.
No Makefile.orig in SRC_w2web, leaving directory.
---------During compiling---------------------
RC_BerryPI ...
make: *** No rule to make target `clean'. Stop.
make: *** No targets specified and no makefile found. Stop.
SRC_clmcopy ...
make: Circular modules.o <- modules.o dependency dropped.
make: Circular reallocate.o <- modules.o dependency dropped.
make: Circular reallocate.o <- reallocate.o dependency dropped.
make[1]: Leaving directory `/usr/local/WIEN2k_14.2/SRC_dstart'
make: *** No rule to make target `complex'. Stop.
-----------------------------------
Waiting for solution.
Thanks!
WT
+++++++++++Here is my situation and installation process++++++++++++++++
1. Hardware: A simple cluster with TWO DELL T760/T7600 workstations
communicating through 1000M switch:
Master (node1)--- 2*Xeon E5-2690 (2*8core), 128G memory
Slave (node2) ---- 2*Xeon E5-2690 v2 (2*10core), 64G memory
---Installation is on Master.
2. OS: CentOS 6.5 64bit final
3. Required package installed
F90 compiler ----Intel parallel_studio_xe_2013_update4 inluding
ifort version 13.1.3, icc version 13.1.3, MKL, installed at /opt/intel
perl 5 ---- v5.10.1 installed at /usr/bin/perl)
emacs --- GNU Emacs 23.1.1 installed at /usr/bin/emacs）
ghostscript ---- v8.70 installed at /usr/bin/ghostscript)
gnuplot ----- gnuplot 4.2 patchlevel 6 installed at /usr/bin/gnuplot ）
www-browser ----- firefox 38.2.1 installed
Tcl/Tk-Toolkit ----- tcl.x86_64/tk.x86_64 v8.5.7 installed
FFTW v.2 or 3 -----Both installed with yum (fftw2-2.1.5, fftw-3.2.1),
and also installed fftw3.3.4 from source through
"./configure --prefix=/usr/local/fftw3.3.4
CC="icc" --enable-mpi --enable-threads"
and "make", "make install" at
/usr/local/fftw3.3.4 with parallel available.
MPI+SCALAPACK -----Yum installed mpich-3.1-4, openmpi-1.8.1,
scalapack-common-2.0.2,
scalapack-openmpi-2.0.2,scalapack-mpich-2.0.2,blacs-common-2.0.2,blacs-mpich-2.0.2,
blacs-openmpi-2.0.2
Also mpich-3.1 was installed from source at
/usr/local/mpich3_intel13 (compiled with ifort13 and icc13
compiler)****This mpich path is indicated in the bashrc and profile*****
Python ---- Python 2.6.6 (with numpy1.4) came with OS,
Python 2.7.3 installed at /usr/local/lib/python2.7,
/usr/local/bin
New Numpy 1.8.0 was uninstalled, and Numpy 1.6.2 was
installed with Python 2.7.3.
Intel MKL-fft ----Intel MKL-fft was compiled through "make
libintel64 F=intel precision=MKL_DOUBLE" at
/opt/intel/mkl/interfaces/xxxxx (xxxx=fftw2xc fftw2xf fftw3xc fftw3xf)
4. Install WIEN2k_14.2
mkdir "WIEN2k_14.2" at /usr/local/
copy Fccni.tar.gz TiC.tar.gz TiO2.tar.gz WIEN2k_14.2.tar to
/usr/local/WIEN2k_14.2, then:
# tar -xvf WIEN2k_14.2.tar
# gunzip *.gz
# chmod +x ./expand_lapw
# ./expand_lapw
***Everything is unpacked except Fccni.tar*****, then
# tar -xf Fccni.tar
# ./siteconfig_lapw
**********************
* Specify a system *
**********************
Selection: I
***********************
* Specify compilers *
***********************
Your Fortran compiler will be ifort.
Your C compiler will be cc.
******************************
* Specify compiler options *
******************************
searching ....
You have the following mkl libraries in
/opt/intel/composer_xe_2013.5.192/mkl/lib/intel64 :
/opt/intel/composer_xe_2013.5.192/mkl/lib/intel64/libmkl_blacs_intelmpi_ilp64.so
......
/opt/intel/composer_xe_2013.5.192/mkl/lib/intel64/libmkl_lapack95_ilp64.a
MKL_TARGET_ARCH was set to intel64
The default options shown on the next screen should be ok
Hit Enter to continue
Current settings:
O Compiler options: -FR -mp1 -w -prec_div -pc80 -pad -ip
-DINTEL_VML -O3 -xW
F FFTW options:
L Linker Flags: $(FOPT)
-L/opt/intel/composer_xe_2013.5.192/mkl/lib/intel64/ -pthread -i-static
P Preprocessor flags '-DParallel'
R R_LIB (LAPACK+BLAS): -lmkl_lapack95_lp64 -lmkl_intel_lp64
-lmkl_intel_thread -lmkl_core -openmp -lpthread
FL FFTW_LIBS: -lfftw3_mpi -lfftw3
-L/usr/local/fftw3_intel12_mpich3/lib
S Save and Quit
*******************************
* Changing compiler options *
*******************************
changing Makefile in SRC_lib/blas_lapw
changing Makefile in SRC_lib/lapack_lapw
No Makefile.orig in SRC_structeditor/SRC_lib, leaving
directory. (*****There are several this statement,No
Makefile.orig****** )
.....
No Makefile.orig in SRC_BerryPI, leaving directory.
......
No Makefile.orig in SRC_lib, leaving directory.
......
No Makefile.orig in SRC_structeditor, leaving directory.
......
No Makefile.orig in SRC_templates, leaving directory.
......
No Makefile.orig in SRC_usersguide_html, leaving directory.
.......
No Makefile.orig in SRC_w2web, leaving directory.
**********************************
* Configure parallel execution *
**********************************
Shared Memory Architecture? (y/N):N
If you are using mpi2 set MPI_REMOTE to 0
Set MPI_REMOTE to 0 / 1: 1
**********************************
* Configure parallel execution *
**********************************
Remote shell (default is ssh) = ssh
********************************************************
You need to KNOW details about your installed MPI and FFTW )
(y/n) y
Your compiler: mpif90
Finding the required fftw2/3 mpi-files in /usr and /opt ....
/usr/local/fftw3.3.4/lib/libfftw3_mpi.a
could not find fftw .... (*************here I don't know
why not found**********)
Your present FFTW choice is:
Please specify whether you want to use FFTW3 (default) or FFTW2
(FFTW3 / FFTW2): FFTW3
Present FFTWpath is:
Please specify the ROOT-path of your FFTW installation (like
/opt/fftw3): /usr/local/fftw3.3.4
Your FFTW_LIBS are: -lfftw3_mpi -lfftw3 -L/usr/local/fftw3.3.4/lib
Your FFTW_OPT are : -DFFTW3 -I/usr/local/fftw3.3.4/include
is this correct? enter Y (default) or n: Y
The blacs-library depends on the version of your mpi (intelmpi, openmpi,
sgimpi, others) and you have to specify the proper blacs-library
in /opt/intel/composer_xe_2013.5.192/mkl/lib/intel64 you have the
following blacs libraries:
/opt/intel/composer_xe_2013.5.192/mkl/lib/intel64/libmkl_blacs_intelmpi_lp64.a
/opt/intel/composer_xe_2013.5.192/mkl/lib/intel64/libmkl_blacs_intelmpi_lp64.so
/opt/intel/composer_xe_2013.5.192/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.a
/opt/intel/composer_xe_2013.5.192/mkl/lib/intel64/libmkl_blacs_sgimpt_lp64.a
Current settings:
FFTW_LIB + FFTW_OPT : -lfftw3_mpi -lfftw3
-L/usr/local/fftw3.3.4/lib + -DFFTW3
-I/usr/local/fftw3.3.4/include (already set)
RP RP_LIB(SCALAPACK+PBLAS): -lmkl_scalapack_lp64
-lmkl_blacs_lp64 $(R_LIBS)
FP FPOPT(par.comp.options): $(FOPT)
MP MPIRUN commando : mpirun -np _NP_ -machinefile
_HOSTS_ _EXEC_
*******************************
* Changing compiler options *
*******************************
changing Makefile in SRC_lapw0
changing Makefile in SRC_lapw1
changing Makefile in SRC_lapw2
changing Makefile in SRC_hf
changing Makefile in SRC_dstart
changing Makefile in SRC_lapwso
changing Makefile in SRC_nmr
***********************************
* (Re-)Dimension parameters *
***********************************
set value for NMATMAX=320000
set value for NUME=32000
************************************
* Compile/Recompile programs *
************************************
A Compile all programs (suggested)
Selection: A
**********I watched the screen and saw a few errors in the
following*****************
RC_BerryPI ...
make: *** No rule to make target `clean'. Stop.
make: *** No targets specified and no makefile found. Stop.
SRC_clmcopy ...
rm -f clmcopy.o modules.o reallocate.o struct_afm_check.o
readstruct.o writestruct.o outerr.o *.mod
ifort -FR -mp1 -w -prec_div -pc80 -pad -ip -DINTEL_VML -O3 -xW -c
clmcopy.f
make: Circular modules.o <- modules.o dependency dropped.
make: Circular reallocate.o <- modules.o dependency dropped.
make: Circular reallocate.o <- reallocate.o dependency dropped.
ifort: command line remark #10010: option '-pthread' is deprecated
and will be removed in a future release. See '-help deprecated'
make[1]: Leaving directory `/usr/local/WIEN2k_14.2/SRC_dstart'
make: *** No rule to make target `complex'. Stop.
......
Copying programs
SRC_afminput/afminput
......
SRC_BerryPI/BerryPI
cp: omitting directory `SRC_BerryPI/BerryPI'
SRC_broadening/broadening
.....
SRC_w2w/w2w
SRC_w2w/w2wc
SRC_w2web/w2web
cp: `SRC_w2web/w2web' and `/usr/local/WIEN2k_14.2/w2web' are the
same file
SRC_wplot/wplot
SRC_wplot/wplotc
SRC_structeditor/SRC_ncmsymmetry/ncmsymmetry
SRC_structeditor/SRC_readwrite/readwrite
SRC_structeditor/SRC_struct2mol/struct2mol
SRC_structeditor/SRC_structgen/structgen
done.
--
Professor Laurence Marks
Department of Materials Science and Engineering
Northwestern University
www.numis.northwestern.edu <http://www.numis.northwestern.edu>
Corrosion in 4D: MURI4D.numis.northwestern.edu
<http://MURI4D.numis.northwestern.edu>
Co-Editor, Acta Cryst A
"Research is to see what everybody else has seen, and to think what
nobody else has thought"
Albert Szent-Gyorgi
_______________________________________________ Wien mailing list
Wien@zeus.theochem.tuwien.ac.at
http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien SEARCH the
MAILING-LIST at:
http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html
_______________________________________________
Wien mailing list
Wien@zeus.theochem.tuwien.ac.at
http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien
SEARCH the MAILING-LIST at:
http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html
--
P.Blaha
--------------------------------------------------------------------------
Peter BLAHA, Inst.f. Materials Chemistry, TU Vienna, A-1060 Vienna
Phone: +43-1-58801-165300 FAX: +43-1-58801-165982
Email: bl...@theochem.tuwien.ac.at WIEN2k: http://www.wien2k.at
WWW: http://www.imc.tuwien.ac.at/staff/tc_group_e.php
--------------------------------------------------------------------------
_______________________________________________
Wien mailing list
Wien@zeus.theochem.tuwien.ac.at
http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien
SEARCH the MAILING-LIST at:
http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html
