Dear Pascal, since you have more than one k-point, you could try to have each pool within one node, so that only inter-pool communication occurs over infiniband; for instance if you have 4 k-points you may try to use 4 pools on 4 nodes (or possibly 2 pools on 2 nodes, or 4 pools on 2 nodes, etc). This kind of parallelization should scale pretty well, if your system allows it (i.e. you have enough kpoints and your system fits in RAM). Then, you can try to optimize the parallelization using the other parallelization options.
If you manage to do some scaling tests using the pools, could you please report your results on this mailing list? Thanks, and best regards, Giovanni Pizzi On 02/05/2013 10:12 PM, pascal boulet wrote: > Dear all, > > I have a basic question about parallelism and scaling factor. > > First, I am running calculations on a cubic system with 58 atoms > (alat=19.5652 a.u.), 540 electrons (324 KS states) and few k-points > (4x4x4 grid=4 k-points), on 32 cores (4 nodes) but I can submit on many > more. > > I guess the best thing to do is to parallelize the calculation on the > bands but maybe also on the FFTs. We have an infiniband interconnection > network between the nodes. > > What would you suggest as values for image/pools/ntg/bands ? > > I have made a SCF test calculation on 16 and 32 cores. For the SCF cycle > (13 steps) I get the following timing: > For 16 cores: total cpu time spent up to now is 22362.4 secs > For 32 cores: total cpu time spent up to now is 17932.6 secs > > The speedup is "only" 25%. I would have expected a better speedup for > such a small number of cores. Am i wrong? What is your experience? > > (For additional information, if helpful: QE 5.0.1 has been compiled with > openMPI, intel 12.1 and FFTW 3.2.2.) > > thank you for your answers. > > Regards > Pascal > > > > _______________________________________________ > Pw_forum mailing list > Pw_forum at pwscf.org > http://pwscf.org/mailman/listinfo/pw_forum -- Giovanni Pizzi Post-doctoral Research Scientist EPFL STI IMX THEOS MXC 340 (B?timent MXC) Station 12 CH-1015 Lausanne (Switzerland) Phone: +41 21 69 31124 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://pwscf.org/pipermail/pw_forum/attachments/20130206/65b493f8/attachment-0001.html
