[Wien] k point parallel in a supercomputer that returns a single name for .machines file
setenv in the job script gave PBS_NODEFILE=/var/spool/PBS/aux/597048.sdb and cat $PBS_NODEFILE in the job script gave gave just a single line nid00136 I gave this problem to our computer people, so hopefully they come with something useful. br, Markus On 23 December 2010 20:26, Peter Blaha wrote: > > What gives: > > echo $PBS_NODEFILE > > Am 23.12.2010 18:53, schrieb Markus Kaukonen: >> >> Thank's for the info, >> >> I think the problem is that in 'my' (finnish taxpayers) system >> cat $PBS_NODEFILE >> gives only a single line and a single node name. >> Also other $PBS related variables did not contain this info. >> So I try to figure out how in Portable Batch System, Professional >> Edition (PBS Pro) >> the true node names are obtained. >> >>> PS: was the test on login6 a 12 core machine ? >> >> It was either 4 or 8 core node, which I did ?not know, because I did >> not select either one. >> But the size of the nodes can be selected by PBS pro, so this is not a >> problem. >> >> Terveisin, Markus >> >> >> >> >> > > -- > - > Peter Blaha > Inst. Materials Chemistry, TU Vienna > Getreidemarkt 9, A-1060 Vienna, Austria > Tel: +43-1-5880115671 > Fax: +43-1-5880115698 > email: pblaha at theochem.tuwien.ac.at > - > ___ > Wien mailing list > Wien at zeus.theochem.tuwien.ac.at > http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien > -- --www=http://www.iki.fi/markus.kaukonen --Markus.Kaukonen at iki.fi --office: N102 Nano building FIN-02015 TKK --home: Viinirinne 3 F 12, 02630 Espoo, FIN --tel: h 045-1242068, w 4518694, 050-5112785 --Rikos ei kannata, eika maatalous --Suomessa. (Paimio 1998) ---
[Wien] k point parallel in a supercomputer that returns a single name for .machines file
Thank's for the info, I think the problem is that in 'my' (finnish taxpayers) system cat $PBS_NODEFILE gives only a single line and a single node name. Also other $PBS related variables did not contain this info. So I try to figure out how in Portable Batch System, Professional Edition (PBS Pro) the true node names are obtained. >PS: was the test on login6 a 12 core machine ? It was either 4 or 8 core node, which I did not know, because I did not select either one. But the size of the nodes can be selected by PBS pro, so this is not a problem. Terveisin, Markus -- --www=http://www.iki.fi/markus.kaukonen --Markus.Kaukonen at iki.fi --office: N102 Nano building FIN-02015 TKK --home: Viinirinne 3 F 12, 02630 Espoo, FIN --tel: h 045-1242068, w 4518694, 050-5112785 --Rikos ei kannata, eika maatalous --Suomessa. (Paimio 1998) ---
[Wien] k point parallel in a supercomputer that returns a single name for .machines file
What gives: echo $PBS_NODEFILE Am 23.12.2010 18:53, schrieb Markus Kaukonen: > Thank's for the info, > > I think the problem is that in 'my' (finnish taxpayers) system > cat $PBS_NODEFILE > gives only a single line and a single node name. > Also other $PBS related variables did not contain this info. > So I try to figure out how in Portable Batch System, Professional > Edition (PBS Pro) > the true node names are obtained. > >> PS: was the test on login6 a 12 core machine ? > It was either 4 or 8 core node, which I did not know, because I did > not select either one. > But the size of the nodes can be selected by PBS pro, so this is not a > problem. > > Terveisin, Markus > > > > > -- - Peter Blaha Inst. Materials Chemistry, TU Vienna Getreidemarkt 9, A-1060 Vienna, Austria Tel: +43-1-5880115671 Fax: +43-1-5880115698 email: pblaha at theochem.tuwien.ac.at -
[Wien] hi
http://nazca_leo.blogsfail.com/10/01/world-has-experienced-a.html -- next part -- An HTML attachment was scrubbed... URL: <http://zeus.theochem.tuwien.ac.at/pipermail/wien/attachments/20101223/d60489be/attachment.htm>
[Wien] k point parallel in a supercomputer that returns a single name for .machines file
Dear Prof. Blaha & Wien2k, On 23 December 2010 12:08, Peter Blaha wrote: > Definitely you should not use such a .machines file. > > If I understand you correctly, using ? #PBS -l mppwidth=14 > you request 14 cores. Why 14 ? I hope you do have 14 k-points > (or a multiple of 14 ???) Yes, I have 14 k-points and would like to run k-parallel job and not MPI parallel jobs for 14 PE. This seems to be not possible in an efficient way for wien2k in machines which have nodes of 2-12 cores. Did I get this right? If so I'd add k-point only parallelism for multicore setups to my wish list, or is there some reason why this would not make sense? > > Now you have to find out in your job script, which nodes you got. > I'd assume, ? ? echo $PBS_NODEFILE > gives you the required info. On the PBS I've used, it would give me a list > of 14 names. This seems to be my problem in this particular system. For some reason $PBS_NODEFILE gives me only one name eventhough there should be at least two node names in this case. I try to sort this out. Merry Christmas from the land of Santa, Markus -- --www=http://www.iki.fi/markus.kaukonen --Markus.Kaukonen at iki.fi --office: N102 Nano building FIN-02015 TKK --home: Viinirinne 3 F 12, 02630 Espoo, FIN --tel: h 045-1242068, w 4518694, 050-5112785 --Rikos ei kannata, eika maatalous --Suomessa. (Paimio 1998) ---
[Wien] k point parallel in a supercomputer that returns a single name for .machines file
> Yes, I have 14 k-points and would like to run k-parallel job and not > MPI parallel jobs for 14 PE. > This seems to be not possible in an efficient way for wien2k in > machines which have nodes of 2-12 cores. > Did I get this right? > If so I'd add k-point only parallelism for multicore setups to my wish > list, or is there some reason why this would not make sense? No, you did not get this right. Wien2k is very well suited for k-point parallelization for multicore setups. In fact, k-point parallelization is always preferred over mpi-parallelization, unless the systems become so big, that they cannot run on one core. In my previous email I only tried to explain, how you could "optimize" the performance on some "bad" and unbalanced systems. (A "bad" system is running a single program X in 100 sec, but when running this program N times simultaneously (N...number of cores) it takes 150 or more seconds, while on a "good" system the increase of cpu-time is smaller (10-20 percent). But with either kind of parallelization YOU must tell which nodes one should use for parallelization. PS: was the test on login6 a 12 core machine ? Your machines file will create 12 parallel jobs with one k-point and once they finish, 2 more jobs with 1 k-point will be scheduled (assuming you have 14 k-points) , while the other cores will idle. Thus in this case I'd use 7 lines, so that only 7 parallel jobs with 2 k-points each are started. (In addition you may specify OMP_NUM_THREAD) Computing is an "art", but there is no way out if you want to use a machine efficiently. > >> >> Now you have to find out in your job script, which nodes you got. >> I'd assume, echo $PBS_NODEFILE >> gives you the required info. On the PBS I've used, it would give me a list >> of 14 names. > This seems to be my problem in this particular system. For some reason > $PBS_NODEFILE gives me only one name eventhough > there should be at least two node names in this case. I try to sort this out. > > Merry Christmas from the land of Santa, > Markus > -- P.Blaha -- Peter BLAHA, Inst.f. Materials Chemistry, TU Vienna, A-1060 Vienna Phone: +43-1-58801-15671 FAX: +43-1-58801-15698 Email: blaha at theochem.tuwien.ac.atWWW: http://info.tuwien.ac.at/theochem/ --
[Wien] k point parallel in a supercomputer that returns a single name for .machines file
Definitely you should not use such a .machines file. If I understand you correctly, using #PBS -l mppwidth=14 you request 14 cores. Why 14 ? I hope you do have 14 k-points (or a multiple of 14 ???) Now you have to find out in your job script, which nodes you got. I'd assume, echo $PBS_NODEFILE gives you the required info. On the PBS I've used, it would give me a list of 14 names. You can also put a command likeenv into your job-file. It should list all defined environment variables and once you have their names, "echo" those which might be from PBS and contain useful info in the next test job. Once you know the names of the nodes (and definitely it should NOT be 12 times the name "login6", you can create a .machines file with 14 lines like: 1:loginX # max 4 times if you have 4 cores/node, or 8 times on the other nodes 1:loginX 1:loginX 1:loginX 1:loginY # next node, repeated until you get 14 such lines. granularity:1 extrafine:1 Of course, the required number of cores must be "commensurate" with your k-points. It does NOT make much sense to request 14 cores, but having 15 k-points. To get a speedup, each lapw1-junk must cost some reasonable time. Otherwise the overhead (jobs are started with one sec delay, more disk-I/O, sumpara,...) will kill you. In other words: don't expect any speedup for the TiC example when running on 12 or 14 nodes. Each single lapw1-part must take at least about 10 sec. Further considerations: Some CPUs are limited by the speed of their memory access. This means that on a 8-core node, eg. one single lapw1 would take 100 sec, but when running 4 lapw1 in parallel, each run would take 130 sec and using 8 lapw1 jobs, each job may even run 200 sec. Obviously, you have to find out the best throughput/ performance for your hardware. Sometimes one can get best performance when using only 4 lapw1 on one 8 node node, but setting OMP_NUM_THREAD to 2 (enabeling mkl-parallelization for ifort/mkl). Am 21.12.2010 15:40, schrieb Markus Kaukonen: > Dear WIEN2k, > > I have a system where there are 4 or 8 cores in one node. > (http://www.csc.fi/english/pages/louhi_guide/hardware/computenodes) > > All cores see the same disk space. > System has Portable Batch System, Professional Edition (PBS Pro) > Typically $PBS_NODEFILE > contains only a name of a single node (or some name for a collection > of nodes???) > > System setup is such that one cannot use > #PBS -l nodes=8:oneproc > (which is probably rude agains other users) but must use > #PBS -l mppwidth=14 > > I would like to run k-point parallel wien2k in this system (and not > mpi parallel). > Is this possible? > To me it seems that if I generate a .machines file > # > 1:login6:1 > 1:login6:1 > 1:login6:1 > 1:login6:1 > 1:login6:1 > 1:login6:1 > 1:login6:1 > 1:login6:1 > 1:login6:1 > 1:login6:1 > 1:login6:1 > 1:login6:1 > granularity:1 > extrafine:1 > > one does not gain in speed when compared to a single core. > > Terveisin, Markus > > -- P.Blaha -- Peter BLAHA, Inst.f. Materials Chemistry, TU Vienna, A-1060 Vienna Phone: +43-1-58801-15671 FAX: +43-1-58801-15698 Email: blaha at theochem.tuwien.ac.atWWW: http://info.tuwien.ac.at/theochem/ --
