subject:"\[gmx\-users\] gromacs on GPU"

As I understood that gromacs version already has included openMM so
the installation of the external openMM sources is not needed, isnt it
?

also I wounder to know what exactly CUDA version is needed ? For
example I've tried lattest cuda-5.0 but with that version i've obtain
error from mdrun-openmm that platform cuda was not detected (gromacs
was compilated without any errors). I've seen that erorr in other
posts in the mailing list but could not found any possible sollutions.

by the way is it possible to compilate gromacs-4.6 agains other
platgorm ( e.g openCL) ? I have no problems with the compatibility of
the openCL and openMM.

James

2013/1/9 Szilárd Páll szilard.p...@cbr.su.se:
 On Tue, Jan 8, 2013 at 3:22 PM, James Starlight jmsstarli...@gmail.comwrote:

 So could someone provide me more about gpu-accelerated MD implemented
 in the 4.6 gromacs ? Does it require openMM (what version is supported


 FYI, if nobody can, trust G:
 http://lmgtfy.com/?q=gromacs+4.6+gpu+acceleration
 http://lmgtfy.com/?q=gromacs+4.6+installation+instructions

 The wiki and mailing list contains quite extensive information (indexed by
 G).

 Otherwise, release notes (not final):
 http://www.gromacs.org/About_Gromacs/Release_Notes/Versions_4.6.x

 Install guide is at the expected location:
 http://www.gromacs.org/Documentation/Installation_Instructions

 Cheers,
 --
 Szilárd


 for that gromacs release ?) installed? By the way at present time I
 force with the problem of compilation 4.1.1 openMM (i need to compile
 openMM because of cuda-5.0 ). If someone have done it (openMM 4.11
 +cuda 5.0 + gromacs-4.5 for lattest geforces) please let me know.


 James

 2013/1/7 James Starlight jmsstarli...@gmail.com:
  Hi Szilárd!
 
  As I understood you correctly gromacs-4.6 have specific algorithm
  (independent on openMM?) for gpu-based calculations havent ? If it
  true how I should compilate such new gpu-based gromacs? In the
  gromacs-4.6-beta-3 folder I've found instructuon for the standart
  installation via cmake
 
  cmake PATH_TO_SOURCE_DIRECTORY -DGMX_OPENMM=ON -DGMX_THREADS=OFF
 
 
  James
 
  2013/1/7 Szilárd Páll szilard.p...@cbr.su.se:
  Szilárd
 --
 gmx-users mailing listgmx-users@gromacs.org
 http://lists.gromacs.org/mailman/listinfo/gmx-users
 * Please search the archive at
 http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
 * Please don't post (un)subscribe requests to the list. Use the
 www interface or send it to gmx-users-requ...@gromacs.org.
 * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

 --
 gmx-users mailing listgmx-users@gromacs.org
 http://lists.gromacs.org/mailman/listinfo/gmx-users
 * Please search the archive at 
 http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
 * Please don't post (un)subscribe requests to the list. Use the
 www interface or send it to gmx-users-requ...@gromacs.org.
 * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] gromacs on GPU

I've solved previous problem but havent been able to launch
mdrun-openmm. Below you can find mdrun's output

Back Off! I just backed up md_test.log to ./#md_test.log.3#
Reading file md_test.tpr, VERSION 4.6-beta3 (single precision)
Changing nstlist from 10 to 40, rlist from 1 to 1.146

Compiled acceleration: None (Gromacs could use AVX_256 on this
machine, which is better)

1 GPU detected:
  #0: NVIDIA GeForce GTX 670, compute cap.: 3.0, ECC:  no, stat: compatible

1 GPU auto-selected to be used for this run: #0

Program mdrun-openmm, VERSION 4.6-beta3
Source code file:
/home/own/gromacs-4.6-beta3/src/kernel/openmm_wrapper.cpp, line: 1367

Fatal error:
OpenMM exception caught while initializating: Error setting device
flags cannot set while device is active in this process
For more information and tips for troubleshooting, please check the GROMACS
website at http://www.gromacs.org/Documentation/Errors


I'm still using cuda-5-toolkit with the 304.54 driver. Also my
question about openCL is still valid. I've browesed manual but didnt
found how I can compile gromacs with openCL instead of Cuda. On openMM
group developers told me that openCL would give much performance for
gpu-accelerated simulation in comparison to the CUDA.

James

2013/1/9 James Starlight jmsstarli...@gmail.com:
 As I understood that gromacs version already has included openMM so
 the installation of the external openMM sources is not needed, isnt it
 ?

 also I wounder to know what exactly CUDA version is needed ? For
 example I've tried lattest cuda-5.0 but with that version i've obtain
 error from mdrun-openmm that platform cuda was not detected (gromacs
 was compilated without any errors). I've seen that erorr in other
 posts in the mailing list but could not found any possible sollutions.

 by the way is it possible to compilate gromacs-4.6 agains other
 platgorm ( e.g openCL) ? I have no problems with the compatibility of
 the openCL and openMM.

 James

 2013/1/9 Szilárd Páll szilard.p...@cbr.su.se:
 On Tue, Jan 8, 2013 at 3:22 PM, James Starlight 
 jmsstarli...@gmail.comwrote:

 So could someone provide me more about gpu-accelerated MD implemented
 in the 4.6 gromacs ? Does it require openMM (what version is supported


 FYI, if nobody can, trust G:
 http://lmgtfy.com/?q=gromacs+4.6+gpu+acceleration
 http://lmgtfy.com/?q=gromacs+4.6+installation+instructions

 The wiki and mailing list contains quite extensive information (indexed by
 G).

 Otherwise, release notes (not final):
 http://www.gromacs.org/About_Gromacs/Release_Notes/Versions_4.6.x

 Install guide is at the expected location:
 http://www.gromacs.org/Documentation/Installation_Instructions

 Cheers,
 --
 Szilárd


 for that gromacs release ?) installed? By the way at present time I
 force with the problem of compilation 4.1.1 openMM (i need to compile
 openMM because of cuda-5.0 ). If someone have done it (openMM 4.11
 +cuda 5.0 + gromacs-4.5 for lattest geforces) please let me know.


 James

 2013/1/7 James Starlight jmsstarli...@gmail.com:
  Hi Szilárd!
 
  As I understood you correctly gromacs-4.6 have specific algorithm
  (independent on openMM?) for gpu-based calculations havent ? If it
  true how I should compilate such new gpu-based gromacs? In the
  gromacs-4.6-beta-3 folder I've found instructuon for the standart
  installation via cmake
 
  cmake PATH_TO_SOURCE_DIRECTORY -DGMX_OPENMM=ON -DGMX_THREADS=OFF
 
 
  James
 
  2013/1/7 Szilárd Páll szilard.p...@cbr.su.se:
  Szilárd
 --
 gmx-users mailing listgmx-users@gromacs.org
 http://lists.gromacs.org/mailman/listinfo/gmx-users
 * Please search the archive at
 http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
 * Please don't post (un)subscribe requests to the list. Use the
 www interface or send it to gmx-users-requ...@gromacs.org.
 * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

 --
 gmx-users mailing listgmx-users@gromacs.org
 http://lists.gromacs.org/mailman/listinfo/gmx-users
 * Please search the archive at 
 http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
 * Please don't post (un)subscribe requests to the list. Use the
 www interface or send it to gmx-users-requ...@gromacs.org.
 * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] gromacs on GPU

2013-01-09 Thread Roland Schulz

On Wed, Jan 9, 2013 at 3:17 AM, James Starlight jmsstarli...@gmail.comwrote:

 As I understood that gromacs version already has included openMM so
 the installation of the external openMM sources is not needed, isnt it
 ?


No the new build in GPU implementation and openMM are two different things.
The Gromacs-OpenMM interface isn't actively maintained and thus not
recommended.


 also I wounder to know what exactly CUDA version is needed ? For
 example I've tried lattest cuda-5.0 but with that version i've obtain
 error from mdrun-openmm that platform cuda was not detected (gromacs
 was compilated without any errors).

with the natived gpu implementation (GMX_GPU) cuda 5.0 works fine.


 by the way is it possible to compilate gromacs-4.6 agains other
 platgorm ( e.g openCL) ? I have no problems with the compatibility of
 the openCL and openMM.

GMX_GPU doesn't support openCL.

Roland



 James

 2013/1/9 Szilárd Páll szilard.p...@cbr.su.se:
  On Tue, Jan 8, 2013 at 3:22 PM, James Starlight jmsstarli...@gmail.com
 wrote:
 
  So could someone provide me more about gpu-accelerated MD implemented
  in the 4.6 gromacs ? Does it require openMM (what version is supported
 
 
  FYI, if nobody can, trust G:
  http://lmgtfy.com/?q=gromacs+4.6+gpu+acceleration
  http://lmgtfy.com/?q=gromacs+4.6+installation+instructions
 
  The wiki and mailing list contains quite extensive information (indexed
 by
  G).
 
  Otherwise, release notes (not final):
  http://www.gromacs.org/About_Gromacs/Release_Notes/Versions_4.6.x
 
  Install guide is at the expected location:
  http://www.gromacs.org/Documentation/Installation_Instructions
 
  Cheers,
  --
  Szilárd
 
 
  for that gromacs release ?) installed? By the way at present time I
  force with the problem of compilation 4.1.1 openMM (i need to compile
  openMM because of cuda-5.0 ). If someone have done it (openMM 4.11
  +cuda 5.0 + gromacs-4.5 for lattest geforces) please let me know.
 
 
  James
 
  2013/1/7 James Starlight jmsstarli...@gmail.com:
   Hi Szilárd!
  
   As I understood you correctly gromacs-4.6 have specific algorithm
   (independent on openMM?) for gpu-based calculations havent ? If it
   true how I should compilate such new gpu-based gromacs? In the
   gromacs-4.6-beta-3 folder I've found instructuon for the standart
   installation via cmake
  
   cmake PATH_TO_SOURCE_DIRECTORY -DGMX_OPENMM=ON -DGMX_THREADS=OFF
  
  
   James
  
   2013/1/7 Szilárd Páll szilard.p...@cbr.su.se:
   Szilárd
  --
  gmx-users mailing listgmx-users@gromacs.org
  http://lists.gromacs.org/mailman/listinfo/gmx-users
  * Please search the archive at
  http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
  * Please don't post (un)subscribe requests to the list. Use the
  www interface or send it to gmx-users-requ...@gromacs.org.
  * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
 
  --
  gmx-users mailing listgmx-users@gromacs.org
  http://lists.gromacs.org/mailman/listinfo/gmx-users
  * Please search the archive at
 http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
  * Please don't post (un)subscribe requests to the list. Use the
  www interface or send it to gmx-users-requ...@gromacs.org.
  * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
 --
 gmx-users mailing listgmx-users@gromacs.org
 http://lists.gromacs.org/mailman/listinfo/gmx-users
 * Please search the archive at
 http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
 * Please don't post (un)subscribe requests to the list. Use the
 www interface or send it to gmx-users-requ...@gromacs.org.
 * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists







-- 
ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
865-241-1537, ORNL PO BOX 2008 MS6309
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] gromacs on GPU

Roland,

indeed the error was that I'have compilate mdrun-openmm which is not
the native gpu.

now I've made mdrun via

cmake CMakeLists.txt -DGMX_GPU=ON -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-5.0

and obtain workable gromacs.

My test system consist of calmodulin (charmm27) solvated in tip3p
water. total size of the system: 67752 atoms.
I have set below values on the mdp file as grompp told me

nstlist = 20
cutoff-scheme = Verlet   ; !not quite sure whai is this!
vdwtype = cut-off


 Now on the mdrun output I have

Using 1 MPI thread
Using 4 OpenMP threads

1 GPU detected:
  #0: NVIDIA GeForce GTX 670, compute cap.: 3.0, ECC:  no, stat: compatible

1 GPU auto-selected to be used for this run: #0

does it mean that all 4 cores of my CPU + gpu are used at same time ?
is there any other ways to increase performance ? ( e.g I'm not quite
sure if open_mpi is used with that build because during compilation
cmake asked me only about fftw3 libs).

Is there any ways to monitor total performance ( e.g separate cpu and
gpu usage ) ?


Thanks for suggestions,

James

2013/1/9 Roland Schulz rol...@utk.edu:
 On Wed, Jan 9, 2013 at 3:17 AM, James Starlight jmsstarli...@gmail.comwrote:

 As I understood that gromacs version already has included openMM so
 the installation of the external openMM sources is not needed, isnt it
 ?


 No the new build in GPU implementation and openMM are two different things.
 The Gromacs-OpenMM interface isn't actively maintained and thus not
 recommended.


 also I wounder to know what exactly CUDA version is needed ? For
 example I've tried lattest cuda-5.0 but with that version i've obtain
 error from mdrun-openmm that platform cuda was not detected (gromacs
 was compilated without any errors).

 with the natived gpu implementation (GMX_GPU) cuda 5.0 works fine.


 by the way is it possible to compilate gromacs-4.6 agains other
 platgorm ( e.g openCL) ? I have no problems with the compatibility of
 the openCL and openMM.

 GMX_GPU doesn't support openCL.

 Roland



 James

 2013/1/9 Szilárd Páll szilard.p...@cbr.su.se:
  On Tue, Jan 8, 2013 at 3:22 PM, James Starlight jmsstarli...@gmail.com
 wrote:
 
  So could someone provide me more about gpu-accelerated MD implemented
  in the 4.6 gromacs ? Does it require openMM (what version is supported
 
 
  FYI, if nobody can, trust G:
  http://lmgtfy.com/?q=gromacs+4.6+gpu+acceleration
  http://lmgtfy.com/?q=gromacs+4.6+installation+instructions
 
  The wiki and mailing list contains quite extensive information (indexed
 by
  G).
 
  Otherwise, release notes (not final):
  http://www.gromacs.org/About_Gromacs/Release_Notes/Versions_4.6.x
 
  Install guide is at the expected location:
  http://www.gromacs.org/Documentation/Installation_Instructions
 
  Cheers,
  --
  Szilárd
 
 
  for that gromacs release ?) installed? By the way at present time I
  force with the problem of compilation 4.1.1 openMM (i need to compile
  openMM because of cuda-5.0 ). If someone have done it (openMM 4.11
  +cuda 5.0 + gromacs-4.5 for lattest geforces) please let me know.
 
 
  James
 
  2013/1/7 James Starlight jmsstarli...@gmail.com:
   Hi Szilárd!
  
   As I understood you correctly gromacs-4.6 have specific algorithm
   (independent on openMM?) for gpu-based calculations havent ? If it
   true how I should compilate such new gpu-based gromacs? In the
   gromacs-4.6-beta-3 folder I've found instructuon for the standart
   installation via cmake
  
   cmake PATH_TO_SOURCE_DIRECTORY -DGMX_OPENMM=ON -DGMX_THREADS=OFF
  
  
   James
  
   2013/1/7 Szilárd Páll szilard.p...@cbr.su.se:
   Szilárd
  --
  gmx-users mailing listgmx-users@gromacs.org
  http://lists.gromacs.org/mailman/listinfo/gmx-users
  * Please search the archive at
  http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
  * Please don't post (un)subscribe requests to the list. Use the
  www interface or send it to gmx-users-requ...@gromacs.org.
  * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
 
  --
  gmx-users mailing listgmx-users@gromacs.org
  http://lists.gromacs.org/mailman/listinfo/gmx-users
  * Please search the archive at
 http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
  * Please don't post (un)subscribe requests to the list. Use the
  www interface or send it to gmx-users-requ...@gromacs.org.
  * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
 --
 gmx-users mailing listgmx-users@gromacs.org
 http://lists.gromacs.org/mailman/listinfo/gmx-users
 * Please search the archive at
 http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
 * Please don't post (un)subscribe requests to the list. Use the
 www interface or send it to gmx-users-requ...@gromacs.org.
 * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists







 --
 ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
 865-241-1537, ORNL PO BOX 2008 MS6309
 --
 gmx-users mailing listgmx-users@gromacs.org

Re: [gmx-users] gromacs on GPU

On Wed, Jan 9, 2013 at 9:17 AM, James Starlight jmsstarli...@gmail.comwrote:

 As I understood that gromacs version already has included openMM so
 the installation of the external openMM sources is not needed, isnt it
 ?


The manual and wiki guides are meant to be read. Although they are a bit
dated, the main body of information is still valid. Please read the
documentation; aka RT(F)M:
http://www.gromacs.org/gpu#Compiling_and_installation_of_GROMACS-GPU_from_source


 also I wounder to know what exactly CUDA version is needed ? For
 example I've tried lattest cuda-5.0 but with that version i've obtain
 error from mdrun-openmm that platform cuda was not detected (gromacs
 was compilated without any errors). I've seen that erorr in other
 posts in the mailing list but could not found any possible sollutions.


It surely works with the (admittedly dated) versions the docs suggest,
you mileage may vary with other versions, but in principle everything
should work with the latest compilers, CUDA, and OpenMM (myself and others
have built and ran successfully).



 by the way is it possible to compilate gromacs-4.6 agains other
 platgorm ( e.g openCL) ? I have no problems with the compatibility of

the openCL and openMM.


No, RTM. OpenMM does have an OpenCL plugin, but that's not supported in
GROMACS, if you want to use it you'll have to write your own code around
the OpenMM library. We
have no resources to implement it ATM, but as GROMACS is OSS anyone is
welcome to contribute it.

And let me reiterate, if you're running explicit solvent you should be
using the native GPU acceleration as you've been told before.

Cheers,
Szilard


 James

 2013/1/9 Szilárd Páll szilard.p...@cbr.su.se:
  On Tue, Jan 8, 2013 at 3:22 PM, James Starlight jmsstarli...@gmail.com
 wrote:
 
  So could someone provide me more about gpu-accelerated MD implemented
  in the 4.6 gromacs ? Does it require openMM (what version is supported
 
 
  FYI, if nobody can, trust G:
  http://lmgtfy.com/?q=gromacs+4.6+gpu+acceleration
  http://lmgtfy.com/?q=gromacs+4.6+installation+instructions
 
  The wiki and mailing list contains quite extensive information (indexed
 by
  G).
 
  Otherwise, release notes (not final):
  http://www.gromacs.org/About_Gromacs/Release_Notes/Versions_4.6.x
 
  Install guide is at the expected location:
  http://www.gromacs.org/Documentation/Installation_Instructions
 
  Cheers,
  --
  Szilárd
 
 
  for that gromacs release ?) installed? By the way at present time I
  force with the problem of compilation 4.1.1 openMM (i need to compile
  openMM because of cuda-5.0 ). If someone have done it (openMM 4.11
  +cuda 5.0 + gromacs-4.5 for lattest geforces) please let me know.
 
 
  James
 
  2013/1/7 James Starlight jmsstarli...@gmail.com:
   Hi Szilárd!
  
   As I understood you correctly gromacs-4.6 have specific algorithm
   (independent on openMM?) for gpu-based calculations havent ? If it
   true how I should compilate such new gpu-based gromacs? In the
   gromacs-4.6-beta-3 folder I've found instructuon for the standart
   installation via cmake
  
   cmake PATH_TO_SOURCE_DIRECTORY -DGMX_OPENMM=ON -DGMX_THREADS=OFF
  
  
   James
  
   2013/1/7 Szilárd Páll szilard.p...@cbr.su.se:
   Szilárd
  --
  gmx-users mailing listgmx-users@gromacs.org
  http://lists.gromacs.org/mailman/listinfo/gmx-users
  * Please search the archive at
  http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
  * Please don't post (un)subscribe requests to the list. Use the
  www interface or send it to gmx-users-requ...@gromacs.org.
  * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
 
  --
  gmx-users mailing listgmx-users@gromacs.org
  http://lists.gromacs.org/mailman/listinfo/gmx-users
  * Please search the archive at
 http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
  * Please don't post (un)subscribe requests to the list. Use the
  www interface or send it to gmx-users-requ...@gromacs.org.
  * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
 --
 gmx-users mailing listgmx-users@gromacs.org
 http://lists.gromacs.org/mailman/listinfo/gmx-users
 * Please search the archive at
 http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
 * Please don't post (un)subscribe requests to the list. Use the
 www interface or send it to gmx-users-requ...@gromacs.org.
 * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] gromacs on GPU

On Wed, Jan 9, 2013 at 10:27 AM, James Starlight jmsstarli...@gmail.comwrote:

 I've solved previous problem but havent been able to launch
 mdrun-openmm. Below you can find mdrun's output

 Back Off! I just backed up md_test.log to ./#md_test.log.3#
 Reading file md_test.tpr, VERSION 4.6-beta3 (single precision)
 Changing nstlist from 10 to 40, rlist from 1 to 1.146

 Compiled acceleration: None (Gromacs could use AVX_256 on this
 machine, which is better)


Have you set the CPU acceleration manually to None? If you did *don't* do
that, let the build system pick what's best for you. However, if you did
not do that, there is some issue with our auto-detection; a log file as a
start would possibly help in identifying the issue.



 1 GPU detected:
   #0: NVIDIA GeForce GTX 670, compute cap.: 3.0, ECC:  no, stat: compatible

 1 GPU auto-selected to be used for this run: #0

 Program mdrun-openmm, VERSION 4.6-beta3
 Source code file:
 /home/own/gromacs-4.6-beta3/src/kernel/openmm_wrapper.cpp, line: 1367

 Fatal error:
 OpenMM exception caught while initializating: Error setting device
 flags cannot set while device is active in this process
 For more information and tips for troubleshooting, please check the GROMACS
 website at http://www.gromacs.org/Documentation/Errors


 I'm still using cuda-5-toolkit with the 304.54 driver. Also my
 question about openCL is still valid. I've browesed manual but didnt
 found how I can compile gromacs with openCL instead of Cuda. On openMM
 group developers told me that openCL would give much performance for
 gpu-accelerated simulation in comparison to the CUDA.

 James

 2013/1/9 James Starlight jmsstarli...@gmail.com:
  As I understood that gromacs version already has included openMM so
  the installation of the external openMM sources is not needed, isnt it
  ?
 
  also I wounder to know what exactly CUDA version is needed ? For
  example I've tried lattest cuda-5.0 but with that version i've obtain
  error from mdrun-openmm that platform cuda was not detected (gromacs
  was compilated without any errors). I've seen that erorr in other
  posts in the mailing list but could not found any possible sollutions.
 
  by the way is it possible to compilate gromacs-4.6 agains other
  platgorm ( e.g openCL) ? I have no problems with the compatibility of
  the openCL and openMM.
 
  James
 
  2013/1/9 Szilárd Páll szilard.p...@cbr.su.se:
  On Tue, Jan 8, 2013 at 3:22 PM, James Starlight jmsstarli...@gmail.com
 wrote:
 
  So could someone provide me more about gpu-accelerated MD implemented
  in the 4.6 gromacs ? Does it require openMM (what version is supported
 
 
  FYI, if nobody can, trust G:
  http://lmgtfy.com/?q=gromacs+4.6+gpu+acceleration
  http://lmgtfy.com/?q=gromacs+4.6+installation+instructions
 
  The wiki and mailing list contains quite extensive information (indexed
 by
  G).
 
  Otherwise, release notes (not final):
  http://www.gromacs.org/About_Gromacs/Release_Notes/Versions_4.6.x
 
  Install guide is at the expected location:
  http://www.gromacs.org/Documentation/Installation_Instructions
 
  Cheers,
  --
  Szilárd
 
 
  for that gromacs release ?) installed? By the way at present time I
  force with the problem of compilation 4.1.1 openMM (i need to compile
  openMM because of cuda-5.0 ). If someone have done it (openMM 4.11
  +cuda 5.0 + gromacs-4.5 for lattest geforces) please let me know.
 
 
  James
 
  2013/1/7 James Starlight jmsstarli...@gmail.com:
   Hi Szilárd!
  
   As I understood you correctly gromacs-4.6 have specific algorithm
   (independent on openMM?) for gpu-based calculations havent ? If it
   true how I should compilate such new gpu-based gromacs? In the
   gromacs-4.6-beta-3 folder I've found instructuon for the standart
   installation via cmake
  
   cmake PATH_TO_SOURCE_DIRECTORY -DGMX_OPENMM=ON -DGMX_THREADS=OFF
  
  
   James
  
   2013/1/7 Szilárd Páll szilard.p...@cbr.su.se:
   Szilárd
  --
  gmx-users mailing listgmx-users@gromacs.org
  http://lists.gromacs.org/mailman/listinfo/gmx-users
  * Please search the archive at
  http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
  * Please don't post (un)subscribe requests to the list. Use the
  www interface or send it to gmx-users-requ...@gromacs.org.
  * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
 
  --
  gmx-users mailing listgmx-users@gromacs.org
  http://lists.gromacs.org/mailman/listinfo/gmx-users
  * Please search the archive at
 http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
  * Please don't post (un)subscribe requests to the list. Use the
  www interface or send it to gmx-users-requ...@gromacs.org.
  * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
 --
 gmx-users mailing listgmx-users@gromacs.org
 http://lists.gromacs.org/mailman/listinfo/gmx-users
 * Please search the archive at
 http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
 * Please don't post (un)subscribe requests to

Re: [gmx-users] gromacs on GPU

Dear James,

On Wed, Jan 9, 2013 at 6:17 PM, James Starlight jmsstarli...@gmail.comwrote:

Roland,

indeed the error was that I'have compilate mdrun-openmm which is not
the native gpu.

now I've made mdrun via

cmake CMakeLists.txt -DGMX_GPU=ON
-DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-5.0

and obtain workable gromacs.

My test system consist of calmodulin (charmm27) solvated in tip3p
water. total size of the system: 67752 atoms.
I have set below values on the mdp file as grompp told me

nstlist = 20
cutoff-scheme = Verlet ; !not quite sure whai is this!

Again RTM:
http://www.gromacs.org/Documentation/Cut-off_schemeshttp://www.gromacs.org/Documentation/Cut-off_schemes?highlight=verlet+scheme

vdwtype = cut-off

Now on the mdrun output I have

Using 1 MPI thread
Using 4 OpenMP threads

1 GPU detected:
#0: NVIDIA GeForce GTX 670, compute cap.: 3.0, ECC: no, stat: compatible

1 GPU auto-selected to be used for this run: #0

does it mean that all 4 cores of my CPU + gpu are used at same time ?

Yes.

is there any other ways to increase performance ? ( e.g I'm not quite
sure if open_mpi is used with that build because during compilation
cmake asked me only about fftw3 libs).

There could be, but I/we can't well without more information on what and
how you compiled and ran. The minimum we need is a log file.

Is there any ways to monitor total performance ( e.g separate cpu and
gpu usage ) ?

See the performance table and summary at the end of the log file. For
further CPU monitoring you can use top, for GPUs the nvidia-smi tool that
comes together with the driver has some monitoring capabilities.

Finally, I would strongly suggest that you check the links I posted in the
first mail. There is plenty of documentation and general description on
the heterogeneous (CPU+GPU) acceleration on the wiki and mailing list; the
google results of the searches I hinted will get you exactly that and more.

Let me reiterate: while questions and general interest are appreciated, the
*minimum* expectation is that you at least read the relevant part of the
documentation and related mailing list posts, especially if this has been
already suggested to you. Please do us and yourself a favor and browse
through the available resources.

Cheers,
--
Szilárd

Thanks for suggestions,

James

2013/1/9 Roland Schulz rol...@utk.edu:
On Wed, Jan 9, 2013 at 3:17 AM, James Starlight jmsstarli...@gmail.com
wrote:

As I understood that gromacs version already has included openMM so
the installation of the external openMM sources is not needed, isnt it
?

No the new build in GPU implementation and openMM are two different
things.
The Gromacs-OpenMM interface isn't actively maintained and thus not
recommended.

also I wounder to know what exactly CUDA version is needed ? For
example I've tried lattest cuda-5.0 but with that version i've obtain
error from mdrun-openmm that platform cuda was not detected (gromacs
was compilated without any errors).

with the natived gpu implementation (GMX_GPU) cuda 5.0 works fine.

by the way is it possible to compilate gromacs-4.6 agains other
platgorm ( e.g openCL) ? I have no problems with the compatibility of
the openCL and openMM.

GMX_GPU doesn't support openCL.

Roland

James

2013/1/9 Szilárd Páll szilard.p...@cbr.su.se:
On Tue, Jan 8, 2013 at 3:22 PM, James Starlight
jmsstarli...@gmail.com
wrote:

So could someone provide me more about gpu-accelerated MD implemented
in the 4.6 gromacs ? Does it require openMM (what version is
supported

FYI, if nobody can, trust G:
http://lmgtfy.com/?q=gromacs+4.6+gpu+acceleration
http://lmgtfy.com/?q=gromacs+4.6+installation+instructions

The wiki and mailing list contains quite extensive information
(indexed
by
G).

Otherwise, release notes (not final):
http://www.gromacs.org/About_Gromacs/Release_Notes/Versions_4.6.x

Install guide is at the expected location:
http://www.gromacs.org/Documentation/Installation_Instructions

Cheers,
--
Szilárd

for that gromacs release ?) installed? By the way at present time I
force with the problem of compilation 4.1.1 openMM (i need to compile
openMM because of cuda-5.0 ). If someone have done it (openMM 4.11
+cuda 5.0 + gromacs-4.5 for lattest geforces) please let me know.

James

2013/1/7 James Starlight jmsstarli...@gmail.com:
Hi Szilárd!

As I understood you correctly gromacs-4.6 have specific algorithm
(independent on openMM?) for gpu-based calculations havent ? If it
true how I should compilate such new gpu-based gromacs? In the
gromacs-4.6-beta-3 folder I've found instructuon for the standart
installation via cmake

cmake PATH_TO_SOURCE_DIRECTORY -DGMX_OPENMM=ON -DGMX_THREADS=OFF

James

2013/1/7 Szilárd Páll szilard.p...@cbr.su.se:
Szilárd
--
gmx-users mailing

Re: [gmx-users] gromacs on GPU

Dear Szilárd, thanks for help again!

2013/1/9 Szilárd Páll szilard.p...@cbr.su.se:


 There could be, but I/we can't well without more information on what and
 how you compiled and ran. The minimum we need is a log file.

I've compilated gromacs 4.6-3 beta via simple


cmake CMakeLists.txt -DGMX_GPU=ON -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-5.0
make
sudo make install

I have not added any special params to the grompp or mdrun.

After that I've run tested simulation of the calmodulin in explicit
water ( 60k atoms ) 100ps and obtain next output

Host: starlight  pid: 21028  nodeid: 0  nnodes:  1
Gromacs version:VERSION 4.6-beta3
Precision:  single
MPI library:thread_mpi
OpenMP support: enabled
GPU support:enabled
invsqrt routine:gmx_software_invsqrt(x)
CPU acceleration:   AVX_256
FFT library:fftw-3.3.2-sse2-avx
Large file support: enabled
RDTSCP usage:   enabled
Built on:   Wed Jan  9 20:44:51 MSK 2013
Built by:   own@starlight [CMAKE]
Build OS/arch:  Linux 3.2.0-2-amd64 x86_64
Build CPU vendor:   GenuineIntel
Build CPU brand:Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz
Build CPU family:   6   Model: 58   Stepping: 9
Build CPU features: aes apic avx clfsh cmov cx8 cx16 f16c htt lahf_lm
mmx msr nonstop_tsc pcid pclmuldq pdcm popcnt pse rdrnd rdtscp sse2
sse3 sse4.1 sse4.2 ssse3 tdt x2apic
C compiler: /usr/bin/gcc GNU gcc (Debian 4.6.3-11) 4.6.3
C compiler flags:   -mavx  -Wextra -Wno-missing-field-initializers
-Wno-sign-compare -Wall -Wno-unused -Wunused-value
-fomit-frame-pointer -funroll-all-loops -fexcess-precision=fast  -O3
-DNDEBUG
C++ compiler:   /usr/bin/c++ GNU c++ (Debian 4.6.3-11) 4.6.3
C++ compiler flags: -mavx  -Wextra -Wno-missing-field-initializers
-Wno-sign-compare -Wall -Wno-unused -Wunused-value
-fomit-frame-pointer -funroll-all-loops -fexcess-precision=fast  -O3
-DNDEBUG
CUDA compiler:  nvcc: NVIDIA (R) Cuda compiler driver;Copyright
(c) 2005-2012 NVIDIA Corporation;Built on
Fri_Sep_21_17:28:58_PDT_2012;Cuda compilation tools, release 5.0,
V0.2.1221
CUDA driver:5.0
CUDA runtime:   5.0



   Core t (s)   Wall t (s)(%)
   Time: 2770.700 1051.927  263.4
 (ns/day)(hour/ns)
Performance:8.2142.922

full log can be found here http://www.sendspace.com/file/inum84


Finally when I check CPU usage I notice that only 1 CPU was full
loaded ( 100%) and 2-4 cores were loaded on only 60% but  gave me
strange results that GPU is not used (I've only monitored temperature
of video card and noticed increase of the temperature up to 65 degrees
)

+--+
| NVIDIA-SMI 4.304.54   Driver Version: 304.54 |
|---+--+--+
| GPU  Name | Bus-IdDisp.  | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap| Memory-Usage | GPU-Util  Compute M. |
|===+==+==|
|   0  GeForce GTX 670  | :02:00.0 N/A |  N/A |
| 38%   63C  N/A N/A /  N/A |   9%  174MB / 2047MB | N/A  Default |
+---+--+--+

+-+
| Compute processes:   GPU Memory |
|  GPU   PID  Process name Usage  |
|=|
|0Not Supported   |
+-+


Thanks for help again,

James
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] gromacs on GPU

2013-01-09 Thread Roland Schulz

Hi,

is this an implicit water calculation? If so it shouldn't use PME.

Roland


On Wed, Jan 9, 2013 at 2:27 PM, James Starlight jmsstarli...@gmail.comwrote:

 Dear Szilárd, thanks for help again!

 2013/1/9 Szilárd Páll szilard.p...@cbr.su.se:

 
  There could be, but I/we can't well without more information on what and
  how you compiled and ran. The minimum we need is a log file.
 
 I've compilated gromacs 4.6-3 beta via simple


 cmake CMakeLists.txt -DGMX_GPU=ON
 -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-5.0
 make
 sudo make install

 I have not added any special params to the grompp or mdrun.

 After that I've run tested simulation of the calmodulin in explicit
 water ( 60k atoms ) 100ps and obtain next output

 Host: starlight  pid: 21028  nodeid: 0  nnodes:  1
 Gromacs version:VERSION 4.6-beta3
 Precision:  single
 MPI library:thread_mpi
 OpenMP support: enabled
 GPU support:enabled
 invsqrt routine:gmx_software_invsqrt(x)
 CPU acceleration:   AVX_256
 FFT library:fftw-3.3.2-sse2-avx
 Large file support: enabled
 RDTSCP usage:   enabled
 Built on:   Wed Jan  9 20:44:51 MSK 2013
 Built by:   own@starlight [CMAKE]
 Build OS/arch:  Linux 3.2.0-2-amd64 x86_64
 Build CPU vendor:   GenuineIntel
 Build CPU brand:Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz
 Build CPU family:   6   Model: 58   Stepping: 9
 Build CPU features: aes apic avx clfsh cmov cx8 cx16 f16c htt lahf_lm
 mmx msr nonstop_tsc pcid pclmuldq pdcm popcnt pse rdrnd rdtscp sse2
 sse3 sse4.1 sse4.2 ssse3 tdt x2apic
 C compiler: /usr/bin/gcc GNU gcc (Debian 4.6.3-11) 4.6.3
 C compiler flags:   -mavx  -Wextra -Wno-missing-field-initializers
 -Wno-sign-compare -Wall -Wno-unused -Wunused-value
 -fomit-frame-pointer -funroll-all-loops -fexcess-precision=fast  -O3
 -DNDEBUG
 C++ compiler:   /usr/bin/c++ GNU c++ (Debian 4.6.3-11) 4.6.3
 C++ compiler flags: -mavx  -Wextra -Wno-missing-field-initializers
 -Wno-sign-compare -Wall -Wno-unused -Wunused-value
 -fomit-frame-pointer -funroll-all-loops -fexcess-precision=fast  -O3
 -DNDEBUG
 CUDA compiler:  nvcc: NVIDIA (R) Cuda compiler driver;Copyright
 (c) 2005-2012 NVIDIA Corporation;Built on
 Fri_Sep_21_17:28:58_PDT_2012;Cuda compilation tools, release 5.0,
 V0.2.1221
 CUDA driver:5.0
 CUDA runtime:   5.0

 

Core t (s)   Wall t (s)(%)
Time: 2770.700 1051.927  263.4
  (ns/day)(hour/ns)
 Performance:8.2142.922

 full log can be found here http://www.sendspace.com/file/inum84


 Finally when I check CPU usage I notice that only 1 CPU was full
 loaded ( 100%) and 2-4 cores were loaded on only 60% but  gave me
 strange results that GPU is not used (I've only monitored temperature
 of video card and noticed increase of the temperature up to 65 degrees
 )

 +--+
 | NVIDIA-SMI 4.304.54   Driver Version: 304.54 |

 |---+--+--+
 | GPU  Name | Bus-IdDisp.  | Volatile Uncorr.
 ECC |
 | Fan  Temp  Perf  Pwr:Usage/Cap| Memory-Usage | GPU-Util  Compute
 M. |

 |===+==+==|
 |   0  GeForce GTX 670  | :02:00.0 N/A |
  N/A |
 | 38%   63C  N/A N/A /  N/A |   9%  174MB / 2047MB | N/A
  Default |

 +---+--+--+


 +-+
 | Compute processes:   GPU
 Memory |
 |  GPU   PID  Process name Usage
|

 |=|
 |0Not Supported
 |

 +-+


 Thanks for help again,

 James
 --
 gmx-users mailing listgmx-users@gromacs.org
 http://lists.gromacs.org/mailman/listinfo/gmx-users
 * Please search the archive at
 http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
 * Please don't post (un)subscribe requests to the list. Use the
 www interface or send it to gmx-users-requ...@gromacs.org.
 * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists







-- 
ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
865-241-1537, ORNL PO BOX 2008 MS6309
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] gromacs on GPU

Hi James,

The build looks mostly fine except that you are using fftw3 compiled with
AVX which is slower than with only SSE (even on AVX-capable CPUs) - you
should have been warned about this at configure-time.

Now, performance-wise everything looks fine except that with a 1.2 nm
cut-off your GPU is not able to keep up with the CPU and finish the
non-bonded work before the CPU is done with Bonded + PME. That's why you
see the Wait GPU taking 20% of the total time and that's also why you see
some cores idling (because for 20% of the run-time thread 0 on core 0
is blocked waiting for the GPU while the rest idle).

As the suggestion at the end of the log file point out, you can consider
using a shorter cut-off which will push more work back to the PME on the
CPU, but whether you can do this it depends on your very problem.

There is one more alternative of running two MPI processes on the GPU
(mpirun -np 2 mdrun -gpu_id 00) and using the -nb gpu_cpu mode which will
execute part of the nonbonded on the CPU, but this might not help.

Cheers,

--
Szilárd


On Wed, Jan 9, 2013 at 8:27 PM, James Starlight jmsstarli...@gmail.comwrote:

 Dear Szilárd, thanks for help again!

 2013/1/9 Szilárd Páll szilard.p...@cbr.su.se:

 
  There could be, but I/we can't well without more information on what and
  how you compiled and ran. The minimum we need is a log file.
 
 I've compilated gromacs 4.6-3 beta via simple


 cmake CMakeLists.txt -DGMX_GPU=ON
 -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-5.0
 make
 sudo make install

 I have not added any special params to the grompp or mdrun.

 After that I've run tested simulation of the calmodulin in explicit
 water ( 60k atoms ) 100ps and obtain next output

 Host: starlight  pid: 21028  nodeid: 0  nnodes:  1
 Gromacs version:VERSION 4.6-beta3
 Precision:  single
 MPI library:thread_mpi
 OpenMP support: enabled
 GPU support:enabled
 invsqrt routine:gmx_software_invsqrt(x)
 CPU acceleration:   AVX_256
 FFT library:fftw-3.3.2-sse2-avx
 Large file support: enabled
 RDTSCP usage:   enabled
 Built on:   Wed Jan  9 20:44:51 MSK 2013
 Built by:   own@starlight [CMAKE]
 Build OS/arch:  Linux 3.2.0-2-amd64 x86_64
 Build CPU vendor:   GenuineIntel
 Build CPU brand:Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz
 Build CPU family:   6   Model: 58   Stepping: 9
 Build CPU features: aes apic avx clfsh cmov cx8 cx16 f16c htt lahf_lm
 mmx msr nonstop_tsc pcid pclmuldq pdcm popcnt pse rdrnd rdtscp sse2
 sse3 sse4.1 sse4.2 ssse3 tdt x2apic
 C compiler: /usr/bin/gcc GNU gcc (Debian 4.6.3-11) 4.6.3
 C compiler flags:   -mavx  -Wextra -Wno-missing-field-initializers
 -Wno-sign-compare -Wall -Wno-unused -Wunused-value
 -fomit-frame-pointer -funroll-all-loops -fexcess-precision=fast  -O3
 -DNDEBUG
 C++ compiler:   /usr/bin/c++ GNU c++ (Debian 4.6.3-11) 4.6.3
 C++ compiler flags: -mavx  -Wextra -Wno-missing-field-initializers
 -Wno-sign-compare -Wall -Wno-unused -Wunused-value
 -fomit-frame-pointer -funroll-all-loops -fexcess-precision=fast  -O3
 -DNDEBUG
 CUDA compiler:  nvcc: NVIDIA (R) Cuda compiler driver;Copyright
 (c) 2005-2012 NVIDIA Corporation;Built on
 Fri_Sep_21_17:28:58_PDT_2012;Cuda compilation tools, release 5.0,
 V0.2.1221
 CUDA driver:5.0
 CUDA runtime:   5.0

 

Core t (s)   Wall t (s)(%)
Time: 2770.700 1051.927  263.4
  (ns/day)(hour/ns)
 Performance:8.2142.922

 full log can be found here http://www.sendspace.com/file/inum84


 Finally when I check CPU usage I notice that only 1 CPU was full
 loaded ( 100%) and 2-4 cores were loaded on only 60% but  gave me
 strange results that GPU is not used (I've only monitored temperature
 of video card and noticed increase of the temperature up to 65 degrees
 )

 +--+
 | NVIDIA-SMI 4.304.54   Driver Version: 304.54 |

 |---+--+--+
 | GPU  Name | Bus-IdDisp.  | Volatile Uncorr.
 ECC |
 | Fan  Temp  Perf  Pwr:Usage/Cap| Memory-Usage | GPU-Util  Compute
 M. |

 |===+==+==|
 |   0  GeForce GTX 670  | :02:00.0 N/A |
  N/A |
 | 38%   63C  N/A N/A /  N/A |   9%  174MB / 2047MB | N/A
  Default |

 +---+--+--+


 +-+
 | Compute processes:   GPU
 Memory |
 |  GPU   PID  Process name Usage
|

 |=|
 |0Not Supported
 |

 +-+

Re: [gmx-users] gromacs on GPU

Szilárd ,

 thanks again for explanation!

Today I've performed some tests on my calmodulin in water system with
different cutt-offs (I've used all cutt-ooffs 1.0 , 0.9 and 0.8
respectually)

Below you can see that the highest performance was in case of 0.8 cut-offs

all cut-offs 1.0
 Force evaluation time GPU/CPU: 6.134 ms/4.700 ms = 1.305

NOTE: The GPU has 20% more load than the CPU. This imbalance causes
  performance loss, consider using a shorter cut-off and a finer PME grid.


   Core t (s)   Wall t (s)(%)
   Time: 1313.420  464.035  283.0
 (ns/day)(hour/ns)
Performance:9.3102.578
Finished mdrun on node 0 Thu Jan 10 09:39:23 2013


all cut-offs 0.9
Force evaluation time GPU/CPU: 4.951 ms/4.675 ms = 1.059

   Core t (s)   Wall t (s)(%)
   Time: 2414.930  856.179  282.1
 (ns/day)(hour/ns)
Performance:   10.0922.378
Finished mdrun on node 0 Thu Jan 10 10:09:52 2013

all cut-offs 0.8
 Force evaluation time GPU/CPU: 4.001 ms/4.659 ms = 0.859

   Core t (s)   Wall t (s)(%)
   Time: 1166.390  413.598  282.0
 (ns/day)(hour/ns)
Performance:   10.4452.298
Finished mdrun on node 0 Thu Jan 10 09:50:33 2013

Also I've noticed that 2-4 CPU cores usage in 2 and 3rd case was only
67%. Is there any other ways to increase performance by means of
neighboor search parameters ( e.g nstlist etc) ?
Might such reduced cut-off be used with the force fields ( e,g charmm)
where initially usage of longest cut-offs have given better results
(e,g in charmm27 and gromos56 I always use 1.2 and 1.4 nm for rvdw,
respectually) ?


James

2013/1/10 Szilárd Páll szilard.p...@cbr.su.se:
 Hi James,

 The build looks mostly fine except that you are using fftw3 compiled with
 AVX which is slower than with only SSE (even on AVX-capable CPUs) - you
 should have been warned about this at configure-time.

 Now, performance-wise everything looks fine except that with a 1.2 nm
 cut-off your GPU is not able to keep up with the CPU and finish the
 non-bonded work before the CPU is done with Bonded + PME. That's why you
 see the Wait GPU taking 20% of the total time and that's also why you see
 some cores idling (because for 20% of the run-time thread 0 on core 0
 is blocked waiting for the GPU while the rest idle).

 As the suggestion at the end of the log file point out, you can consider
 using a shorter cut-off which will push more work back to the PME on the
 CPU, but whether you can do this it depends on your very problem.

 There is one more alternative of running two MPI processes on the GPU
 (mpirun -np 2 mdrun -gpu_id 00) and using the -nb gpu_cpu mode which will
 execute part of the nonbonded on the CPU, but this might not help.

 Cheers,

 --
 Szilárd


 On Wed, Jan 9, 2013 at 8:27 PM, James Starlight jmsstarli...@gmail.comwrote:

 Dear Szilárd, thanks for help again!

 2013/1/9 Szilárd Páll szilard.p...@cbr.su.se:

 
  There could be, but I/we can't well without more information on what and
  how you compiled and ran. The minimum we need is a log file.
 
 I've compilated gromacs 4.6-3 beta via simple


 cmake CMakeLists.txt -DGMX_GPU=ON
 -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-5.0
 make
 sudo make install

 I have not added any special params to the grompp or mdrun.

 After that I've run tested simulation of the calmodulin in explicit
 water ( 60k atoms ) 100ps and obtain next output

 Host: starlight  pid: 21028  nodeid: 0  nnodes:  1
 Gromacs version:VERSION 4.6-beta3
 Precision:  single
 MPI library:thread_mpi
 OpenMP support: enabled
 GPU support:enabled
 invsqrt routine:gmx_software_invsqrt(x)
 CPU acceleration:   AVX_256
 FFT library:fftw-3.3.2-sse2-avx
 Large file support: enabled
 RDTSCP usage:   enabled
 Built on:   Wed Jan  9 20:44:51 MSK 2013
 Built by:   own@starlight [CMAKE]
 Build OS/arch:  Linux 3.2.0-2-amd64 x86_64
 Build CPU vendor:   GenuineIntel
 Build CPU brand:Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz
 Build CPU family:   6   Model: 58   Stepping: 9
 Build CPU features: aes apic avx clfsh cmov cx8 cx16 f16c htt lahf_lm
 mmx msr nonstop_tsc pcid pclmuldq pdcm popcnt pse rdrnd rdtscp sse2
 sse3 sse4.1 sse4.2 ssse3 tdt x2apic
 C compiler: /usr/bin/gcc GNU gcc (Debian 4.6.3-11) 4.6.3
 C compiler flags:   -mavx  -Wextra -Wno-missing-field-initializers
 -Wno-sign-compare -Wall -Wno-unused -Wunused-value
 -fomit-frame-pointer -funroll-all-loops -fexcess-precision=fast  -O3
 -DNDEBUG
 C++ compiler:   /usr/bin/c++ GNU c++ (Debian 4.6.3-11) 4.6.3
 C++ compiler flags: -mavx  -Wextra -Wno-missing-field-initializers
 -Wno-sign-compare -Wall -Wno-unused -Wunused-value
 -fomit-frame-pointer -funroll-all-loops -fexcess-precision=fast  -O3
 -DNDEBUG
 CUDA compiler:  nvcc: NVIDIA (R) Cuda compiler driver;Copyright
 (c) 2005-2012 NVIDIA

Re: [gmx-users] gromacs on GPU

2013-01-08 Thread James Starlight

So could someone provide me more about gpu-accelerated MD implemented
in the 4.6 gromacs ? Does it require openMM (what version is supported
for that gromacs release ?) installed? By the way at present time I
force with the problem of compilation 4.1.1 openMM (i need to compile
openMM because of cuda-5.0 ). If someone have done it (openMM 4.11
+cuda 5.0 + gromacs-4.5 for lattest geforces) please let me know.


James

2013/1/7 James Starlight jmsstarli...@gmail.com:
 Hi Szilárd!

 As I understood you correctly gromacs-4.6 have specific algorithm
 (independent on openMM?) for gpu-based calculations havent ? If it
 true how I should compilate such new gpu-based gromacs? In the
 gromacs-4.6-beta-3 folder I've found instructuon for the standart
 installation via cmake

 cmake PATH_TO_SOURCE_DIRECTORY -DGMX_OPENMM=ON -DGMX_THREADS=OFF


 James

 2013/1/7 Szilárd Páll szilard.p...@cbr.su.se:
 Szilárd
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] gromacs on GPU

2013-01-08 Thread Szilárd Páll

On Tue, Jan 8, 2013 at 3:22 PM, James Starlight jmsstarli...@gmail.comwrote:

 So could someone provide me more about gpu-accelerated MD implemented
 in the 4.6 gromacs ? Does it require openMM (what version is supported


FYI, if nobody can, trust G:
http://lmgtfy.com/?q=gromacs+4.6+gpu+acceleration
http://lmgtfy.com/?q=gromacs+4.6+installation+instructions

The wiki and mailing list contains quite extensive information (indexed by
G).

Otherwise, release notes (not final):
http://www.gromacs.org/About_Gromacs/Release_Notes/Versions_4.6.x

Install guide is at the expected location:
http://www.gromacs.org/Documentation/Installation_Instructions

Cheers,
--
Szilárd


 for that gromacs release ?) installed? By the way at present time I
 force with the problem of compilation 4.1.1 openMM (i need to compile
 openMM because of cuda-5.0 ). If someone have done it (openMM 4.11
 +cuda 5.0 + gromacs-4.5 for lattest geforces) please let me know.


 James

 2013/1/7 James Starlight jmsstarli...@gmail.com:
  Hi Szilárd!
 
  As I understood you correctly gromacs-4.6 have specific algorithm
  (independent on openMM?) for gpu-based calculations havent ? If it
  true how I should compilate such new gpu-based gromacs? In the
  gromacs-4.6-beta-3 folder I've found instructuon for the standart
  installation via cmake
 
  cmake PATH_TO_SOURCE_DIRECTORY -DGMX_OPENMM=ON -DGMX_THREADS=OFF
 
 
  James
 
  2013/1/7 Szilárd Páll szilard.p...@cbr.su.se:
  Szilárd
 --
 gmx-users mailing listgmx-users@gromacs.org
 http://lists.gromacs.org/mailman/listinfo/gmx-users
 * Please search the archive at
 http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
 * Please don't post (un)subscribe requests to the list. Use the
 www interface or send it to gmx-users-requ...@gromacs.org.
 * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] gromacs on GPU

2013-01-07 Thread Szilárd Páll

Hi James,

Don't use the OpenMM accelerated mdrun for explicit solvent simulations.
For such runs any modern CPU with 4-8 cores will probably be as fast or
faster than most GPUs.

However, do consider the 4.6 *native* heterogeneous GPU-accelerated code.
This new efficient code can provide 2-4x speedup and uses both CPU and GPU.

To get the latest 4.6 version use the 4.6-beta3 release (http://goo.gl/XFPoe)
or even better, get the code from git (git clone git://
git.gromacs.org/gromacs -b release-4-6).

Let us know how things worked out!

Cheers,

--
Szilárd

On Mon, Jan 7, 2013 at 8:05 AM, James Starlight jmsstarli...@gmail.comwrote:

Today I've tried to test gpu performance on the benchmark system (
explicit solvent with pme )

grompp-gpu -f md_sd.mdp -c conf -p topol.top -o md_test
mdrun-gpu -device
OpenMM:platform=Cuda,DeviceID=1,Memtest=15,force-device=no -v
-deffnm md_test

that produce the error

Program mdrun-gpu, VERSION 4.5.5
Source code file: /home/own/gromacs_gpu/src/kernel/openmm_wrapper.cpp,
line: 1365

Fatal error:
OpenMM exception caught while initializating: cudaMemcpyToSymbol:
SetSim copy to cSim failed invalid device symbol
For more information and tips for troubleshooting, please check the GROMACS
website at http://www.gromacs.org/Documentation/Errors

In the mailing list archive I've found the same error with the same
card posted some time ago

http://article.gmane.org/gmane.science.biology.gromacs.user/55577/match=openmm+exception+caught+while+initializating+cudaMemcpyToSymbol+setsim+copy+cSim+failed+invalid+device+symbol

From that topic I've understood that 670 graphic card is not supported
yet in Gromacs (4.55). I've checked the manual for the 4.6 beta
version and does not found anything about supporting of that card in
the lattest gromacs realeses as well. IS there any ways to solve that
problem ? Could that error be linked with some others ? E.g I've
installed nvidia drivers + cuda first and openMM 4.01 than. During
gromacs compilation I've not forced with any errors.

Thanks for help
James

2013/1/6 Justin Lemkul jalem...@vt.edu:

On 1/6/13 1:53 PM, James Starlight wrote:

OK!

I've compilated gromacs-gpu from the source using that tutorial for
the Debian OS

http://verahill.blogspot.ru/2012/01/debian-testing-64-wheezy-compiling_20.html

The only thing that it lack is the installation mdrun_d-gpu but i'm
not sure that double precission can be used with gpu.

Could someone provide me with some tutorial which would show me basic
mdrun options with the calculation on my gpu as well as some testing
system?

Everything you need to know (including benchmark systems) is posted at
http://www.gromacs.org/Documentation/Installation_Instructions_4.5/GPUs.

The invocation of mdrun-gpu is like any other mdrun, with GPU-specific
information available by reading mdrun-gpu -h.

-Justin

Justin A. Lemkul, Ph.D.
Research Scientist
Department of Biochemistry
Virginia Tech
Blacksburg, VA
jalemkul[at]vt.edu | (540) 231-9080
http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin

--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the www
interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] gromacs on GPU

2013-01-07 Thread James Starlight

Hi Szilárd!

As I understood you correctly gromacs-4.6 have specific algorithm
(independent on openMM?) for gpu-based calculations havent ? If it
true how I should compilate such new gpu-based gromacs? In the
gromacs-4.6-beta-3 folder I've found instructuon for the standart
installation via cmake

cmake PATH_TO_SOURCE_DIRECTORY -DGMX_OPENMM=ON -DGMX_THREADS=OFF


James

2013/1/7 Szilárd Páll szilard.p...@cbr.su.se:
 Szilárd
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

[gmx-users] gromacs on GPU

Dear Gromacs Users!


I want to install gromacs 4.55 on my Debian OS with the GPU support
(I'd like to make some calculations on the GeForce 670 gpu).


I have cuda-toolkit as well as openMM 4.01 installed from the binaries
as well as gromac's sources. Could you provide me with some tutorial
where I could found such step-by-step installation of the Gromacs with
gpu.

Thanks for help


James
-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] gromacs on GPU

2013-01-06 Thread Justin Lemkul




On 1/6/13 4:48 AM, James Starlight wrote:

Dear Gromacs Users!


I want to install gromacs 4.55 on my Debian OS with the GPU support
(I'd like to make some calculations on the GeForce 670 gpu).


I have cuda-toolkit as well as openMM 4.01 installed from the binaries
as well as gromac's sources. Could you provide me with some tutorial
where I could found such step-by-step installation of the Gromacs with
gpu.



http://www.gromacs.org/Documentation/Installation_Instructions_4.5/GPUs#Installing_and_running_GROMACS-GPU

-Justin

--


Justin A. Lemkul, Ph.D.
Research Scientist
Department of Biochemistry
Virginia Tech
Blacksburg, VA
jalemkul[at]vt.edu | (540) 231-9080
http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin


--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] gromacs on GPU

Have someone tried to use gromacs-4.5-gpu binaries ? Would that
packages work on debian or its better to build from sources ? :)

James

2013/1/6 Justin Lemkul jalem...@vt.edu:


 On 1/6/13 4:48 AM, James Starlight wrote:

 Dear Gromacs Users!


 I want to install gromacs 4.55 on my Debian OS with the GPU support
 (I'd like to make some calculations on the GeForce 670 gpu).


 I have cuda-toolkit as well as openMM 4.01 installed from the binaries
 as well as gromac's sources. Could you provide me with some tutorial
 where I could found such step-by-step installation of the Gromacs with
 gpu.


 http://www.gromacs.org/Documentation/Installation_Instructions_4.5/GPUs#Installing_and_running_GROMACS-GPU

 -Justin

 --
 

 Justin A. Lemkul, Ph.D.
 Research Scientist
 Department of Biochemistry
 Virginia Tech
 Blacksburg, VA
 jalemkul[at]vt.edu | (540) 231-9080
 http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin

 
 --
 gmx-users mailing listgmx-users@gromacs.org
 http://lists.gromacs.org/mailman/listinfo/gmx-users
 * Please search the archive at
 http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
 * Please don't post (un)subscribe requests to the list. Use the www
 interface or send it to gmx-users-requ...@gromacs.org.
 * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] gromacs on GPU

2013-01-06 Thread Justin Lemkul




On 1/6/13 7:41 AM, James Starlight wrote:

Have someone tried to use gromacs-4.5-gpu binaries ? Would that
packages work on debian or its better to build from sources ? :)



The binaries online are very outdated.  Compiling from source is rather easy, 
and the website below has all the steps listed.


-Justin

--


Justin A. Lemkul, Ph.D.
Research Scientist
Department of Biochemistry
Virginia Tech
Blacksburg, VA
jalemkul[at]vt.edu | (540) 231-9080
http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin


--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] gromacs on GPU

OK!

I've compilated gromacs-gpu from the source using that tutorial for
the Debian OS
http://verahill.blogspot.ru/2012/01/debian-testing-64-wheezy-compiling_20.html

The only thing that it lack is the installation mdrun_d-gpu  but i'm
not sure that double precission can be used with gpu.

Could someone provide me with some tutorial which would show me basic
mdrun options with the calculation on my gpu as well as some testing
system?

James

2013/1/6 Justin Lemkul jalem...@vt.edu:


 On 1/6/13 7:41 AM, James Starlight wrote:

 Have someone tried to use gromacs-4.5-gpu binaries ? Would that
 packages work on debian or its better to build from sources ? :)


 The binaries online are very outdated.  Compiling from source is rather
 easy, and the website below has all the steps listed.


 -Justin

 --
 

 Justin A. Lemkul, Ph.D.
 Research Scientist
 Department of Biochemistry
 Virginia Tech
 Blacksburg, VA
 jalemkul[at]vt.edu | (540) 231-9080
 http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin

 
 --
 gmx-users mailing listgmx-users@gromacs.org
 http://lists.gromacs.org/mailman/listinfo/gmx-users
 * Please search the archive at
 http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
 * Please don't post (un)subscribe requests to the list. Use the www
 interface or send it to gmx-users-requ...@gromacs.org.
 * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] gromacs on GPU

2013-01-06 Thread Justin Lemkul




On 1/6/13 1:53 PM, James Starlight wrote:

OK!

I've compilated gromacs-gpu from the source using that tutorial for
the Debian OS
http://verahill.blogspot.ru/2012/01/debian-testing-64-wheezy-compiling_20.html

The only thing that it lack is the installation mdrun_d-gpu  but i'm
not sure that double precission can be used with gpu.

Could someone provide me with some tutorial which would show me basic
mdrun options with the calculation on my gpu as well as some testing
system?



Everything you need to know (including benchmark systems) is posted at 
http://www.gromacs.org/Documentation/Installation_Instructions_4.5/GPUs.


The invocation of mdrun-gpu is like any other mdrun, with GPU-specific 
information available by reading mdrun-gpu -h.


-Justin

--


Justin A. Lemkul, Ph.D.
Research Scientist
Department of Biochemistry
Virginia Tech
Blacksburg, VA
jalemkul[at]vt.edu | (540) 231-9080
http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin


--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] gromacs on GPU