Re: [gmx-users] Gomacs and multiple GPU
Hi, Others software doesn't assist the user as gromacs, they don't use CUDA_VISIBLE without an action by the user (with an option, or a specific environnement variable), in case the use this variable they wrote a bit more information, like what they detect and will be used. So i was convince gromacs works in the same way even though you wrote in your mail. sorry :( Alexis On 01/12/16 18:28, Szilárd Páll wrote: > Hi, > > Welcome! > >> Thank for you reply. Yes, i have read them and know them. I use them > will success with other software. > > BTW, what did you refer to here? It seems like you were focused on > figuring out the detection issue and might have ignored my other > recommendations on using multi (which incidentally would have solved > the original issue too as well as would do correct job placement for > you) and 2+ ranks per GPU? > Cheers, > -- > Szilárd > > > On Thu, Dec 1, 2016 at 6:12 PM, Alexis Michon wrote: >> Hi, >> >> You solve my problem, Thank you. >> >> The variable CUDA_VISIBLE_DEVICES is set up by the scheduler at the >> running time of my job. So after adding commandline "unset >> $CUDA_VISIBLE_DEVICES" in my script, mdrun detect and use both GPUs. >> >> Thank you for your patience. >> >> Cheers, >> Alexis >> >> On 01/12/16 17:53, Szilárd Páll wrote: >>> Hi, >>> >>> Do any other programs, e.g. CUDA SDK samples (like deviceQuery) detect >>> both GPUs? >>> I find it *very-very* unlikely that only mdrun would not be able >>> detect both GPUs. >>> >>> Again, are you sure it's not the use of CUDA_VISIBLE_DEVICES that >>> creates the confusion? Please share your full command line (with env >>> vars an all) and resulting log output? >>> >>> Cheers, >>> -- >>> Szilárd >>> >>> >>> On Thu, Dec 1, 2016 at 5:10 PM, Alexis Michon wrote: Hi, Thank for you reply. Yes, i have read them and know them. I use them will success with other software. My first and main problem is mdrun doesn't see the second GPU on the system. And i try to find the root cause of and the remediation. Hardware is a bi xeon with 2 Titan-xp GPU, so my goal is to run 2 differentes simulations with two mdrun process each with one dedicated GPU. Do you have any idea ? Cheers Alexis On 01/12/16 16:38, Szilárd Páll wrote: > Alexis, > > What is the problem? Have you read the relevant docs of what > CUDA_VISIBLE_DEVICES does? > https://docs.nvidia.com/cuda/cuda-c-programming-guide/#env-vars > > > BTW, you can use, but you *do not need* CUDA_VISIBLE devices to > control the mapping of GPU(s) to mdrun process, the -gpu_id variable > (or equivalent env var) alone is enough. > > To run multiple processes per node, do consider: > - using mdrun -multi > - using 2 or more mdrun runs per GPU > > You can find the reason, motivating performance examples, and sample > commands here (in particular Fig 5 and related section): > dx.doi.org/10.1002/jcc.24030 > > Cheers, > -- > Szilárd > > > On Thu, Dec 1, 2016 at 3:38 PM, Alexis Michon > wrote: >> Hi, >> >> thank you for your answer >> >> the Cmdline is: >> gmx mdrun -deffnm nvt >> >> I played a bit with option -gpu_id, use $CUDA_VISIBLE_DEVICES and read >> the page http://www.gromacs.org/GPU_acceleration without any succes. One >> output is in nvt.log.11, >> >> nvidia-smi returs is in nvidia-smi.log >> >> Nvidia-smi detect 2 GPUs, but not mdrun. >> >> nvidia-smi.log : https://icloud.ibcp.fr/index.php/s/08jqFsZpWNlSUGh >> nvt.log : https://icloud.ibcp.fr/index.php/s/vC5UxCCAhyw9FuC >> >> Alexis >> >> >> >> On 01/12/16 11:34, Mark Abraham wrote: >>> Hi, >>> >>> On Wed, Nov 30, 2016 at 5:38 PM Alexis Michon >>> wrote: >>> Hello, We have build gromacs 2016.1 from source with "DMX_GPU=on" on a bi processor bi GPU machine, mdrun detect and run fine on only 1 GPU. How could we force mdrun to detect the second GPU ? >>> If it's compatible, powered and supported by your driver, then mdrun >>> will >>> find it. Presumably nvidia-smi tool will help you work out what's going >>> on. >>> >>> We would like to run 2 mdrun instances on a machine equiped with 2 GPUs, each mdrun will use 1GPU. How could we tell mdrun to use a specific GPU ? >>> Guidance is here >>> http://manual.gromacs.org/documentation/2016.1/user-guide/mdrun-performance.html#examples-for-mdrun-on-one-node >>> >>> Mark >>> >>> Cheers, Alexis -- Citation : "It’s not enough to be busy; so are the ants. The question is: what are we busy about?" - Henry David Thoreau Alexis MICHON, responsable informatique CNRS IBCP, 7 passage du
Re: [gmx-users] Gomacs and multiple GPU
Hi, Welcome! > Thank for you reply. Yes, i have read them and know them. I use them will success with other software. BTW, what did you refer to here? It seems like you were focused on figuring out the detection issue and might have ignored my other recommendations on using multi (which incidentally would have solved the original issue too as well as would do correct job placement for you) and 2+ ranks per GPU? Cheers, -- Szilárd On Thu, Dec 1, 2016 at 6:12 PM, Alexis Michon wrote: > Hi, > > You solve my problem, Thank you. > > The variable CUDA_VISIBLE_DEVICES is set up by the scheduler at the > running time of my job. So after adding commandline "unset > $CUDA_VISIBLE_DEVICES" in my script, mdrun detect and use both GPUs. > > Thank you for your patience. > > Cheers, > Alexis > > On 01/12/16 17:53, Szilárd Páll wrote: >> Hi, >> >> Do any other programs, e.g. CUDA SDK samples (like deviceQuery) detect >> both GPUs? >> I find it *very-very* unlikely that only mdrun would not be able >> detect both GPUs. >> >> Again, are you sure it's not the use of CUDA_VISIBLE_DEVICES that >> creates the confusion? Please share your full command line (with env >> vars an all) and resulting log output? >> >> Cheers, >> -- >> Szilárd >> >> >> On Thu, Dec 1, 2016 at 5:10 PM, Alexis Michon wrote: >>> Hi, >>> >>> Thank for you reply. Yes, i have read them and know them. I use them >>> will success with other software. >>> >>> My first and main problem is mdrun doesn't see the second GPU on the >>> system. And i try to find the root cause of and the remediation. >>> >>> Hardware is a bi xeon with 2 Titan-xp GPU, so my goal is to run 2 >>> differentes simulations with two mdrun process each with one dedicated GPU. >>> >>> Do you have any idea ? >>> >>> Cheers >>> Alexis >>> >>> On 01/12/16 16:38, Szilárd Páll wrote: Alexis, What is the problem? Have you read the relevant docs of what CUDA_VISIBLE_DEVICES does? https://docs.nvidia.com/cuda/cuda-c-programming-guide/#env-vars BTW, you can use, but you *do not need* CUDA_VISIBLE devices to control the mapping of GPU(s) to mdrun process, the -gpu_id variable (or equivalent env var) alone is enough. To run multiple processes per node, do consider: - using mdrun -multi - using 2 or more mdrun runs per GPU You can find the reason, motivating performance examples, and sample commands here (in particular Fig 5 and related section): dx.doi.org/10.1002/jcc.24030 Cheers, -- Szilárd On Thu, Dec 1, 2016 at 3:38 PM, Alexis Michon wrote: > Hi, > > thank you for your answer > > the Cmdline is: > gmx mdrun -deffnm nvt > > I played a bit with option -gpu_id, use $CUDA_VISIBLE_DEVICES and read > the page http://www.gromacs.org/GPU_acceleration without any succes. One > output is in nvt.log.11, > > nvidia-smi returs is in nvidia-smi.log > > Nvidia-smi detect 2 GPUs, but not mdrun. > > nvidia-smi.log : https://icloud.ibcp.fr/index.php/s/08jqFsZpWNlSUGh > nvt.log : https://icloud.ibcp.fr/index.php/s/vC5UxCCAhyw9FuC > > Alexis > > > > On 01/12/16 11:34, Mark Abraham wrote: >> Hi, >> >> On Wed, Nov 30, 2016 at 5:38 PM Alexis Michon >> wrote: >> >>> Hello, >>> >>> We have build gromacs 2016.1 from source with "DMX_GPU=on" on a bi >>> processor bi GPU machine, mdrun detect and run fine on only 1 GPU. How >>> could we force mdrun to detect the second GPU ? >>> >> If it's compatible, powered and supported by your driver, then mdrun will >> find it. Presumably nvidia-smi tool will help you work out what's going >> on. >> >> We would like to run 2 mdrun instances on a machine equiped with 2 GPUs, >>> each mdrun will use 1GPU. How could we tell mdrun to use a specific >>> GPU ? >>> >> Guidance is here >> http://manual.gromacs.org/documentation/2016.1/user-guide/mdrun-performance.html#examples-for-mdrun-on-one-node >> >> Mark >> >> >>> Cheers, >>> Alexis >>> >>> -- >>> Citation : "It’s not enough to be busy; so are the ants. The question >>> is: >>> what are we busy about?" - Henry David Thoreau >>> Alexis MICHON, responsable informatique >>> CNRS IBCP, 7 passage du vercors, 69007 LYON, France >>> Mail : alexis.mic...@ibcp.fr Tel : 04.72.72.26.03 - 06.27.56.34.80 >>> CNRS IBCP - UMS 5760 - http://www.ibcp.fr/ >>> Empreinte : C9:45:2D:7C:79:7F:0B:79:CA:C8:0B:68:41:A2:8C:EE:EA:72:82:34 >>> >>> >>> -- >>> Gromacs Users mailing list >>> >>> * Please search the archive at >>> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before >>> posting! >>> >>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists >>> >>> * For (un)subscribe requests visit >>> https://mail
Re: [gmx-users] Gomacs and multiple GPU
Hi, You solve my problem, Thank you. The variable CUDA_VISIBLE_DEVICES is set up by the scheduler at the running time of my job. So after adding commandline "unset $CUDA_VISIBLE_DEVICES" in my script, mdrun detect and use both GPUs. Thank you for your patience. Cheers, Alexis On 01/12/16 17:53, Szilárd Páll wrote: > Hi, > > Do any other programs, e.g. CUDA SDK samples (like deviceQuery) detect > both GPUs? > I find it *very-very* unlikely that only mdrun would not be able > detect both GPUs. > > Again, are you sure it's not the use of CUDA_VISIBLE_DEVICES that > creates the confusion? Please share your full command line (with env > vars an all) and resulting log output? > > Cheers, > -- > Szilárd > > > On Thu, Dec 1, 2016 at 5:10 PM, Alexis Michon wrote: >> Hi, >> >> Thank for you reply. Yes, i have read them and know them. I use them >> will success with other software. >> >> My first and main problem is mdrun doesn't see the second GPU on the >> system. And i try to find the root cause of and the remediation. >> >> Hardware is a bi xeon with 2 Titan-xp GPU, so my goal is to run 2 >> differentes simulations with two mdrun process each with one dedicated GPU. >> >> Do you have any idea ? >> >> Cheers >> Alexis >> >> On 01/12/16 16:38, Szilárd Páll wrote: >>> Alexis, >>> >>> What is the problem? Have you read the relevant docs of what >>> CUDA_VISIBLE_DEVICES does? >>> https://docs.nvidia.com/cuda/cuda-c-programming-guide/#env-vars >>> >>> >>> BTW, you can use, but you *do not need* CUDA_VISIBLE devices to >>> control the mapping of GPU(s) to mdrun process, the -gpu_id variable >>> (or equivalent env var) alone is enough. >>> >>> To run multiple processes per node, do consider: >>> - using mdrun -multi >>> - using 2 or more mdrun runs per GPU >>> >>> You can find the reason, motivating performance examples, and sample >>> commands here (in particular Fig 5 and related section): >>> dx.doi.org/10.1002/jcc.24030 >>> >>> Cheers, >>> -- >>> Szilárd >>> >>> >>> On Thu, Dec 1, 2016 at 3:38 PM, Alexis Michon wrote: Hi, thank you for your answer the Cmdline is: gmx mdrun -deffnm nvt I played a bit with option -gpu_id, use $CUDA_VISIBLE_DEVICES and read the page http://www.gromacs.org/GPU_acceleration without any succes. One output is in nvt.log.11, nvidia-smi returs is in nvidia-smi.log Nvidia-smi detect 2 GPUs, but not mdrun. nvidia-smi.log : https://icloud.ibcp.fr/index.php/s/08jqFsZpWNlSUGh nvt.log : https://icloud.ibcp.fr/index.php/s/vC5UxCCAhyw9FuC Alexis On 01/12/16 11:34, Mark Abraham wrote: > Hi, > > On Wed, Nov 30, 2016 at 5:38 PM Alexis Michon > wrote: > >> Hello, >> >> We have build gromacs 2016.1 from source with "DMX_GPU=on" on a bi >> processor bi GPU machine, mdrun detect and run fine on only 1 GPU. How >> could we force mdrun to detect the second GPU ? >> > If it's compatible, powered and supported by your driver, then mdrun will > find it. Presumably nvidia-smi tool will help you work out what's going > on. > > We would like to run 2 mdrun instances on a machine equiped with 2 GPUs, >> each mdrun will use 1GPU. How could we tell mdrun to use a specific GPU >> ? >> > Guidance is here > http://manual.gromacs.org/documentation/2016.1/user-guide/mdrun-performance.html#examples-for-mdrun-on-one-node > > Mark > > >> Cheers, >> Alexis >> >> -- >> Citation : "It’s not enough to be busy; so are the ants. The question is: >> what are we busy about?" - Henry David Thoreau >> Alexis MICHON, responsable informatique >> CNRS IBCP, 7 passage du vercors, 69007 LYON, France >> Mail : alexis.mic...@ibcp.fr Tel : 04.72.72.26.03 - 06.27.56.34.80 >> CNRS IBCP - UMS 5760 - http://www.ibcp.fr/ >> Empreinte : C9:45:2D:7C:79:7F:0B:79:CA:C8:0B:68:41:A2:8C:EE:EA:72:82:34 >> >> >> -- >> Gromacs Users mailing list >> >> * Please search the archive at >> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before >> posting! >> >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists >> >> * For (un)subscribe requests visit >> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or >> send a mail to gmx-users-requ...@gromacs.org. -- Citation : "It’s not enough to be busy; so are the ants. The question is: what are we busy about?" - Henry David Thoreau Alexis MICHON, responsable informatique CNRS IBCP, 7 passage du vercors, 69007 LYON, France Mail : alexis.mic...@ibcp.fr Tel : 04.72.72.26.03 - 06.27.56.34.80 CNRS IBCP - UMS 5760 - http://www.ibcp.fr/ Empreinte : C9:45:2D:7C:79:7F:0B:79:CA:C8:0B:68:41:A2:8C:EE:EA:72:82:34 -- Gromacs Users mailing list * Please search the archive at http://ww
Re: [gmx-users] Gomacs and multiple GPU
Hi, Do any other programs, e.g. CUDA SDK samples (like deviceQuery) detect both GPUs? I find it *very-very* unlikely that only mdrun would not be able detect both GPUs. Again, are you sure it's not the use of CUDA_VISIBLE_DEVICES that creates the confusion? Please share your full command line (with env vars an all) and resulting log output? Cheers, -- Szilárd On Thu, Dec 1, 2016 at 5:10 PM, Alexis Michon wrote: > Hi, > > Thank for you reply. Yes, i have read them and know them. I use them > will success with other software. > > My first and main problem is mdrun doesn't see the second GPU on the > system. And i try to find the root cause of and the remediation. > > Hardware is a bi xeon with 2 Titan-xp GPU, so my goal is to run 2 > differentes simulations with two mdrun process each with one dedicated GPU. > > Do you have any idea ? > > Cheers > Alexis > > On 01/12/16 16:38, Szilárd Páll wrote: >> Alexis, >> >> What is the problem? Have you read the relevant docs of what >> CUDA_VISIBLE_DEVICES does? >> https://docs.nvidia.com/cuda/cuda-c-programming-guide/#env-vars >> >> >> BTW, you can use, but you *do not need* CUDA_VISIBLE devices to >> control the mapping of GPU(s) to mdrun process, the -gpu_id variable >> (or equivalent env var) alone is enough. >> >> To run multiple processes per node, do consider: >> - using mdrun -multi >> - using 2 or more mdrun runs per GPU >> >> You can find the reason, motivating performance examples, and sample >> commands here (in particular Fig 5 and related section): >> dx.doi.org/10.1002/jcc.24030 >> >> Cheers, >> -- >> Szilárd >> >> >> On Thu, Dec 1, 2016 at 3:38 PM, Alexis Michon wrote: >>> Hi, >>> >>> thank you for your answer >>> >>> the Cmdline is: >>> gmx mdrun -deffnm nvt >>> >>> I played a bit with option -gpu_id, use $CUDA_VISIBLE_DEVICES and read >>> the page http://www.gromacs.org/GPU_acceleration without any succes. One >>> output is in nvt.log.11, >>> >>> nvidia-smi returs is in nvidia-smi.log >>> >>> Nvidia-smi detect 2 GPUs, but not mdrun. >>> >>> nvidia-smi.log : https://icloud.ibcp.fr/index.php/s/08jqFsZpWNlSUGh >>> nvt.log : https://icloud.ibcp.fr/index.php/s/vC5UxCCAhyw9FuC >>> >>> Alexis >>> >>> >>> >>> On 01/12/16 11:34, Mark Abraham wrote: Hi, On Wed, Nov 30, 2016 at 5:38 PM Alexis Michon wrote: > Hello, > > We have build gromacs 2016.1 from source with "DMX_GPU=on" on a bi > processor bi GPU machine, mdrun detect and run fine on only 1 GPU. How > could we force mdrun to detect the second GPU ? > If it's compatible, powered and supported by your driver, then mdrun will find it. Presumably nvidia-smi tool will help you work out what's going on. We would like to run 2 mdrun instances on a machine equiped with 2 GPUs, > each mdrun will use 1GPU. How could we tell mdrun to use a specific GPU ? > Guidance is here http://manual.gromacs.org/documentation/2016.1/user-guide/mdrun-performance.html#examples-for-mdrun-on-one-node Mark > Cheers, > Alexis > > -- > Citation : "It’s not enough to be busy; so are the ants. The question is: > what are we busy about?" - Henry David Thoreau > Alexis MICHON, responsable informatique > CNRS IBCP, 7 passage du vercors, 69007 LYON, France > Mail : alexis.mic...@ibcp.fr Tel : 04.72.72.26.03 - 06.27.56.34.80 > CNRS IBCP - UMS 5760 - http://www.ibcp.fr/ > Empreinte : C9:45:2D:7C:79:7F:0B:79:CA:C8:0B:68:41:A2:8C:EE:EA:72:82:34 > > > -- > Gromacs Users mailing list > > * Please search the archive at > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before > posting! > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > * For (un)subscribe requests visit > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or > send a mail to gmx-users-requ...@gromacs.org. >>> -- >>> Citation : "It’s not enough to be busy; so are the ants. The question is: >>> what are we busy about?" - Henry David Thoreau >>> Alexis MICHON, responsable informatique >>> CNRS IBCP, 7 passage du vercors, 69007 LYON, France >>> Mail : alexis.mic...@ibcp.fr Tel : 04.72.72.26.03 - 06.27.56.34.80 >>> CNRS IBCP - UMS 5760 - http://www.ibcp.fr/ >>> Empreinte : C9:45:2D:7C:79:7F:0B:79:CA:C8:0B:68:41:A2:8C:EE:EA:72:82:34 >>> >>> >>> >>> -- >>> Gromacs Users mailing list >>> >>> * Please search the archive at >>> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! >>> >>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists >>> >>> * For (un)subscribe requests visit >>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send >>> a mail to gmx-users-requ...@gromacs.org. > > -- > Citation : "It’s not enough to be busy; so are the ants. The question is: > what are we busy about?" - Henry David Thoreau > Alexis MICHON, responsable informatique > CNRS IBCP,
Re: [gmx-users] Gomacs and multiple GPU
Hi, Thank for you reply. Yes, i have read them and know them. I use them will success with other software. My first and main problem is mdrun doesn't see the second GPU on the system. And i try to find the root cause of and the remediation. Hardware is a bi xeon with 2 Titan-xp GPU, so my goal is to run 2 differentes simulations with two mdrun process each with one dedicated GPU. Do you have any idea ? Cheers Alexis On 01/12/16 16:38, Szilárd Páll wrote: > Alexis, > > What is the problem? Have you read the relevant docs of what > CUDA_VISIBLE_DEVICES does? > https://docs.nvidia.com/cuda/cuda-c-programming-guide/#env-vars > > > BTW, you can use, but you *do not need* CUDA_VISIBLE devices to > control the mapping of GPU(s) to mdrun process, the -gpu_id variable > (or equivalent env var) alone is enough. > > To run multiple processes per node, do consider: > - using mdrun -multi > - using 2 or more mdrun runs per GPU > > You can find the reason, motivating performance examples, and sample > commands here (in particular Fig 5 and related section): > dx.doi.org/10.1002/jcc.24030 > > Cheers, > -- > Szilárd > > > On Thu, Dec 1, 2016 at 3:38 PM, Alexis Michon wrote: >> Hi, >> >> thank you for your answer >> >> the Cmdline is: >> gmx mdrun -deffnm nvt >> >> I played a bit with option -gpu_id, use $CUDA_VISIBLE_DEVICES and read >> the page http://www.gromacs.org/GPU_acceleration without any succes. One >> output is in nvt.log.11, >> >> nvidia-smi returs is in nvidia-smi.log >> >> Nvidia-smi detect 2 GPUs, but not mdrun. >> >> nvidia-smi.log : https://icloud.ibcp.fr/index.php/s/08jqFsZpWNlSUGh >> nvt.log : https://icloud.ibcp.fr/index.php/s/vC5UxCCAhyw9FuC >> >> Alexis >> >> >> >> On 01/12/16 11:34, Mark Abraham wrote: >>> Hi, >>> >>> On Wed, Nov 30, 2016 at 5:38 PM Alexis Michon wrote: >>> Hello, We have build gromacs 2016.1 from source with "DMX_GPU=on" on a bi processor bi GPU machine, mdrun detect and run fine on only 1 GPU. How could we force mdrun to detect the second GPU ? >>> If it's compatible, powered and supported by your driver, then mdrun will >>> find it. Presumably nvidia-smi tool will help you work out what's going on. >>> >>> We would like to run 2 mdrun instances on a machine equiped with 2 GPUs, each mdrun will use 1GPU. How could we tell mdrun to use a specific GPU ? >>> Guidance is here >>> http://manual.gromacs.org/documentation/2016.1/user-guide/mdrun-performance.html#examples-for-mdrun-on-one-node >>> >>> Mark >>> >>> Cheers, Alexis -- Citation : "It’s not enough to be busy; so are the ants. The question is: what are we busy about?" - Henry David Thoreau Alexis MICHON, responsable informatique CNRS IBCP, 7 passage du vercors, 69007 LYON, France Mail : alexis.mic...@ibcp.fr Tel : 04.72.72.26.03 - 06.27.56.34.80 CNRS IBCP - UMS 5760 - http://www.ibcp.fr/ Empreinte : C9:45:2D:7C:79:7F:0B:79:CA:C8:0B:68:41:A2:8C:EE:EA:72:82:34 -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org. >> -- >> Citation : "It’s not enough to be busy; so are the ants. The question is: >> what are we busy about?" - Henry David Thoreau >> Alexis MICHON, responsable informatique >> CNRS IBCP, 7 passage du vercors, 69007 LYON, France >> Mail : alexis.mic...@ibcp.fr Tel : 04.72.72.26.03 - 06.27.56.34.80 >> CNRS IBCP - UMS 5760 - http://www.ibcp.fr/ >> Empreinte : C9:45:2D:7C:79:7F:0B:79:CA:C8:0B:68:41:A2:8C:EE:EA:72:82:34 >> >> >> >> -- >> Gromacs Users mailing list >> >> * Please search the archive at >> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! >> >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists >> >> * For (un)subscribe requests visit >> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a >> mail to gmx-users-requ...@gromacs.org. -- Citation : "It’s not enough to be busy; so are the ants. The question is: what are we busy about?" - Henry David Thoreau Alexis MICHON, responsable informatique CNRS IBCP, 7 passage du vercors, 69007 LYON, France Mail : alexis.mic...@ibcp.fr Tel : 04.72.72.26.03 - 06.27.56.34.80 CNRS IBCP - UMS 5760 - http://www.ibcp.fr/ Empreinte : C9:45:2D:7C:79:7F:0B:79:CA:C8:0B:68:41:A2:8C:EE:EA:72:82:34 -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.or
Re: [gmx-users] Gomacs and multiple GPU
Alexis, What is the problem? Have you read the relevant docs of what CUDA_VISIBLE_DEVICES does? https://docs.nvidia.com/cuda/cuda-c-programming-guide/#env-vars BTW, you can use, but you *do not need* CUDA_VISIBLE devices to control the mapping of GPU(s) to mdrun process, the -gpu_id variable (or equivalent env var) alone is enough. To run multiple processes per node, do consider: - using mdrun -multi - using 2 or more mdrun runs per GPU You can find the reason, motivating performance examples, and sample commands here (in particular Fig 5 and related section): dx.doi.org/10.1002/jcc.24030 Cheers, -- Szilárd On Thu, Dec 1, 2016 at 3:38 PM, Alexis Michon wrote: > Hi, > > thank you for your answer > > the Cmdline is: > gmx mdrun -deffnm nvt > > I played a bit with option -gpu_id, use $CUDA_VISIBLE_DEVICES and read > the page http://www.gromacs.org/GPU_acceleration without any succes. One > output is in nvt.log.11, > > nvidia-smi returs is in nvidia-smi.log > > Nvidia-smi detect 2 GPUs, but not mdrun. > > nvidia-smi.log : https://icloud.ibcp.fr/index.php/s/08jqFsZpWNlSUGh > nvt.log : https://icloud.ibcp.fr/index.php/s/vC5UxCCAhyw9FuC > > Alexis > > > > On 01/12/16 11:34, Mark Abraham wrote: >> Hi, >> >> On Wed, Nov 30, 2016 at 5:38 PM Alexis Michon wrote: >> >>> Hello, >>> >>> We have build gromacs 2016.1 from source with "DMX_GPU=on" on a bi >>> processor bi GPU machine, mdrun detect and run fine on only 1 GPU. How >>> could we force mdrun to detect the second GPU ? >>> >> If it's compatible, powered and supported by your driver, then mdrun will >> find it. Presumably nvidia-smi tool will help you work out what's going on. >> >> We would like to run 2 mdrun instances on a machine equiped with 2 GPUs, >>> each mdrun will use 1GPU. How could we tell mdrun to use a specific GPU ? >>> >> Guidance is here >> http://manual.gromacs.org/documentation/2016.1/user-guide/mdrun-performance.html#examples-for-mdrun-on-one-node >> >> Mark >> >> >>> Cheers, >>> Alexis >>> >>> -- >>> Citation : "It’s not enough to be busy; so are the ants. The question is: >>> what are we busy about?" - Henry David Thoreau >>> Alexis MICHON, responsable informatique >>> CNRS IBCP, 7 passage du vercors, 69007 LYON, France >>> Mail : alexis.mic...@ibcp.fr Tel : 04.72.72.26.03 - 06.27.56.34.80 >>> CNRS IBCP - UMS 5760 - http://www.ibcp.fr/ >>> Empreinte : C9:45:2D:7C:79:7F:0B:79:CA:C8:0B:68:41:A2:8C:EE:EA:72:82:34 >>> >>> >>> -- >>> Gromacs Users mailing list >>> >>> * Please search the archive at >>> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before >>> posting! >>> >>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists >>> >>> * For (un)subscribe requests visit >>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or >>> send a mail to gmx-users-requ...@gromacs.org. > > -- > Citation : "It’s not enough to be busy; so are the ants. The question is: > what are we busy about?" - Henry David Thoreau > Alexis MICHON, responsable informatique > CNRS IBCP, 7 passage du vercors, 69007 LYON, France > Mail : alexis.mic...@ibcp.fr Tel : 04.72.72.26.03 - 06.27.56.34.80 > CNRS IBCP - UMS 5760 - http://www.ibcp.fr/ > Empreinte : C9:45:2D:7C:79:7F:0B:79:CA:C8:0B:68:41:A2:8C:EE:EA:72:82:34 > > > > -- > Gromacs Users mailing list > > * Please search the archive at > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > * For (un)subscribe requests visit > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a > mail to gmx-users-requ...@gromacs.org. -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org.
Re: [gmx-users] Gomacs and multiple GPU
Hi, thank you for your answer the Cmdline is: gmx mdrun -deffnm nvt I played a bit with option -gpu_id, use $CUDA_VISIBLE_DEVICES and read the page http://www.gromacs.org/GPU_acceleration without any succes. One output is in nvt.log.11, nvidia-smi returs is in nvidia-smi.log Nvidia-smi detect 2 GPUs, but not mdrun. nvidia-smi.log : https://icloud.ibcp.fr/index.php/s/08jqFsZpWNlSUGh nvt.log : https://icloud.ibcp.fr/index.php/s/vC5UxCCAhyw9FuC Alexis On 01/12/16 11:34, Mark Abraham wrote: > Hi, > > On Wed, Nov 30, 2016 at 5:38 PM Alexis Michon wrote: > >> Hello, >> >> We have build gromacs 2016.1 from source with "DMX_GPU=on" on a bi >> processor bi GPU machine, mdrun detect and run fine on only 1 GPU. How >> could we force mdrun to detect the second GPU ? >> > If it's compatible, powered and supported by your driver, then mdrun will > find it. Presumably nvidia-smi tool will help you work out what's going on. > > We would like to run 2 mdrun instances on a machine equiped with 2 GPUs, >> each mdrun will use 1GPU. How could we tell mdrun to use a specific GPU ? >> > Guidance is here > http://manual.gromacs.org/documentation/2016.1/user-guide/mdrun-performance.html#examples-for-mdrun-on-one-node > > Mark > > >> Cheers, >> Alexis >> >> -- >> Citation : "It’s not enough to be busy; so are the ants. The question is: >> what are we busy about?" - Henry David Thoreau >> Alexis MICHON, responsable informatique >> CNRS IBCP, 7 passage du vercors, 69007 LYON, France >> Mail : alexis.mic...@ibcp.fr Tel : 04.72.72.26.03 - 06.27.56.34.80 >> CNRS IBCP - UMS 5760 - http://www.ibcp.fr/ >> Empreinte : C9:45:2D:7C:79:7F:0B:79:CA:C8:0B:68:41:A2:8C:EE:EA:72:82:34 >> >> >> -- >> Gromacs Users mailing list >> >> * Please search the archive at >> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before >> posting! >> >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists >> >> * For (un)subscribe requests visit >> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or >> send a mail to gmx-users-requ...@gromacs.org. -- Citation : "It’s not enough to be busy; so are the ants. The question is: what are we busy about?" - Henry David Thoreau Alexis MICHON, responsable informatique CNRS IBCP, 7 passage du vercors, 69007 LYON, France Mail : alexis.mic...@ibcp.fr Tel : 04.72.72.26.03 - 06.27.56.34.80 CNRS IBCP - UMS 5760 - http://www.ibcp.fr/ Empreinte : C9:45:2D:7C:79:7F:0B:79:CA:C8:0B:68:41:A2:8C:EE:EA:72:82:34 -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org.
Re: [gmx-users] Gomacs and multiple GPU
Hi, On Wed, Nov 30, 2016 at 5:38 PM Alexis Michon wrote: > Hello, > > We have build gromacs 2016.1 from source with "DMX_GPU=on" on a bi > processor bi GPU machine, mdrun detect and run fine on only 1 GPU. How > could we force mdrun to detect the second GPU ? > If it's compatible, powered and supported by your driver, then mdrun will find it. Presumably nvidia-smi tool will help you work out what's going on. We would like to run 2 mdrun instances on a machine equiped with 2 GPUs, > each mdrun will use 1GPU. How could we tell mdrun to use a specific GPU ? > Guidance is here http://manual.gromacs.org/documentation/2016.1/user-guide/mdrun-performance.html#examples-for-mdrun-on-one-node Mark > Cheers, > Alexis > > -- > Citation : "It’s not enough to be busy; so are the ants. The question is: > what are we busy about?" - Henry David Thoreau > Alexis MICHON, responsable informatique > CNRS IBCP, 7 passage du vercors, 69007 LYON, France > Mail : alexis.mic...@ibcp.fr Tel : 04.72.72.26.03 - 06.27.56.34.80 > CNRS IBCP - UMS 5760 - http://www.ibcp.fr/ > Empreinte : C9:45:2D:7C:79:7F:0B:79:CA:C8:0B:68:41:A2:8C:EE:EA:72:82:34 > > > -- > Gromacs Users mailing list > > * Please search the archive at > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before > posting! > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > * For (un)subscribe requests visit > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or > send a mail to gmx-users-requ...@gromacs.org. -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org.
Re: [gmx-users] Gomacs and multiple GPU
Hi, It would be better if you provide more details such as the mdrun command and log file etc. -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org.
