Hi,
How can we designate which CPU-only nodes to be PME-dedicated nodes?
What mdrun options or what configuration should we use to make that happen?
BR,
Theo
On 8/11/2014 9:36 PM, Mark Abraham wrote:
Hi,
What Carsten said, if running on nodes that have GPUs.
If running on a mixed setup (some nodes with GPU, some not), then arranging
your MPI environment to place PME ranks on CPU-only nodes is probably
worthwhile. For example, all your PP ranks first, mapped to GPU nodes, then
all your PME ranks, mapped to CPU-only nodes, and then use mdrun -ddorder
pp_pme.
Mark
On Mon, Aug 11, 2014 at 2:45 AM, Theodore Si <sjyz...@gmail.com> wrote:
Hi Mark,
This is information of our cluster, could you give us some advice as
regards to our cluster so that we can make GMX run faster on our system?
Each CPU node has 2 CPUs and each GPU node has 2 CPUs and 2 Nvidia K20M
Device Name Device Type Specifications Number
CPU Node IntelH2216JFFKRNodes CPU: 2×Intel Xeon E5-2670(8 Cores,
2.6GHz, 20MB Cache, 8.0GT)
Mem: 64GB(8×8GB) ECC Registered DDR3 1600MHz Samsung Memory 332
Fat Node IntelH2216WPFKRNodes CPU: 2×Intel Xeon E5-2670(8 Cores,
2.6GHz, 20MB Cache, 8.0GT)
Mem: 256G(16×16G) ECC Registered DDR3 1600MHz Samsung Memory 20
GPU Node IntelR2208GZ4GC CPU: 2×Intel Xeon E5-2670(8 Cores,
2.6GHz, 20MB Cache, 8.0GT)
Mem: 64GB(8×8GB) ECC Registered DDR3 1600MHz Samsung Memory 50
MIC Node IntelR2208GZ4GC CPU: 2×Intel Xeon E5-2670(8 Cores,
2.6GHz, 20MB Cache, 8.0GT)
Mem: 64GB(8×8GB) ECC Registered DDR3 1600MHz Samsung Memory 5
Computing Network Switch Mellanox Infiniband FDR Core Switch
648× FDR Core Switch MSX6536-10R, Mellanox Unified Fabric Manager 1
Mellanox SX1036 40Gb Switch 36× 40Gb Ethernet Switch SX1036, 36× QSFP
Interface 1
Management Network Switch Extreme Summit X440-48t-10G 2-layer Switch
48× 1Giga Switch Summit X440-48t-10G, authorized by ExtremeXOS 9
Extreme Summit X650-24X 3-layer Switch 24× 10Giga 3-layer Ethernet Switch
Summit X650-24X, authorized by ExtremeXOS 1
Parallel Storage DDN Parallel Storage System DDN SFA12K Storage
System 1
GPU GPU Accelerator NVIDIA Tesla Kepler K20M 70
MIC MIC Intel Xeon Phi 5110P Knights Corner 10
40Gb Ethernet Card MCX314A-BCBT Mellanox ConnextX-3 Chip 40Gb
Ethernet Card
2× 40Gb Ethernet ports, enough QSFP cables 16
SSD Intel SSD910 Intel SSD910 Disk, 400GB, PCIE 80
On 8/10/2014 5:50 AM, Mark Abraham wrote:
That's not what I said.... "You can set..."
-npme behaves the same whether or not GPUs are in use. Using separate
ranks
for PME caters to trying to minimize the cost of the all-to-all
communication of the 3DFFT. That's still relevant when using GPUs, but if
separate PME ranks are used, any GPUs on nodes that only have PME ranks
are
left idle. The most effective approach depends critically on the hardware
and simulation setup, and whether you pay money for your hardware.
Mark
On Sat, Aug 9, 2014 at 2:56 AM, Theodore Si <sjyz...@gmail.com> wrote:
Hi,
You mean no matter we use GPU acceleration or not, -npme is just a
reference?
Why we can't set that to a exact value?
On 8/9/2014 5:14 AM, Mark Abraham wrote:
You can set the number of PME-only ranks with -npme. Whether it's useful
is
another matter :-) The CPU-based PME offload and the GPU-based PP
offload
do not combine very well.
Mark
On Fri, Aug 8, 2014 at 7:24 AM, Theodore Si <sjyz...@gmail.com> wrote:
Hi,
Can we set the number manually with -npme when using GPU acceleration?
--
Gromacs Users mailing list
* Please search the archive at http://www.gromacs.org/
Support/Mailing_Lists/GMX-Users_List before posting!
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
* For (un)subscribe requests visit
https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
send a mail to gmx-users-requ...@gromacs.org.
--
Gromacs Users mailing list
* Please search the archive at http://www.gromacs.org/
Support/Mailing_Lists/GMX-Users_List before posting!
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
* For (un)subscribe requests visit
https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
send a mail to gmx-users-requ...@gromacs.org.
--
Gromacs Users mailing list
* Please search the archive at http://www.gromacs.org/
Support/Mailing_Lists/GMX-Users_List before posting!
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
* For (un)subscribe requests visit
https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
send a mail to gmx-users-requ...@gromacs.org.
--
Gromacs Users mailing list
* Please search the archive at
http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting!
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
* For (un)subscribe requests visit
https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a
mail to gmx-users-requ...@gromacs.org.
