Hi, Am 27.08.2011 um 16:55 schrieb Gowtham:
> One of the clusters we have at Michigan Tech has 3 compute > nodes, each with 2 nVidia GPU cards (GeForce GTX 260) which > in turn have 216 CUDA cores each. Each of these 3 compute > nodes have 4 CPUs. > > I followed Jon Forrest's instructions here: > > http://marc.info/?l=npaci-rocks-discussion&m=129668495229097&w=2 > > I got it work with > > devdriver_4.0_linux_64_270.41.19.run > cudatoolkit_4.0.17_linux_64_rhel5.5.run > gpucomputingsdk_4.0.17_linux.run > > with very little modifications to the instructions in the > above URL. After setting the necessary variables in user's > .bashrc, I could compile the GPU Computing SDK and when run > > deviceQuery > > on compute nodes (either by logging into compute node & > running it OR submitting a simple SGE job) and get desired, > meaningful results about both GPU cards: > > ... > Found 2 CUDA Capable device(s) > Device 0: "GeForce GTX 260" > CUDA Driver Version / Runtime Version 4.0 / 4.0 > CUDA Capability Major/Minor version number: 1.3 > ... > Device 1: "GeForce GTX 260" > CUDA Driver Version / Runtime Version 4.0 / 4.0 > CUDA Capability Major/Minor version number: 1.3 > ... > deviceQuery, CUDA Driver = CUDART, > CUDA Driver Version = 4.0, > CUDA Runtime Version = 4.0, NumDevs = 2, > Device = GeForce GTX 260, Device = GeForce GTX 260 > ... > > > I followed that chain of emails in the above URL and from > Dung Do's note: > > http://marc.info/?l=npaci-rocks-discussion&m=129675227702026&w=2 when you introduce already a forced compley attached to a queue, you only need to request the forced complex. Specifying the queue in addition is superfluous, it won't hurt though. > I understand that it's possible to control usage of GPU > cores from SGE. I have done the following a few times before > > - creating a new queue > - creating host list and assigning it to a queue > > but what I have not done are > > - create a forced complex (cudaonly) You checked the Wiki: http://wikis.sun.com/display/gridengine62u5/How+to+Configure+the+Complex+From+the+Command+Line > - use nvidia-smi to change all GPU to compute exclusive How does it relate to SGE - when I get it right it's an interface to the GPU, but the exclusive access needs to be defined already beforehand in SGE. To me it doesn't look necessary to set it in nvidia-smi too. It's in the above Wiki http://wikis.sun.com/display/gridengine62u5/Configuring+Exclusive+Scheduling > Following that email further, I too would like to just use > 3 CPU cores from each compute nodes leaving 1 CPU core for > non-GPU jobs. Just define proper slot counts in the queue definition. -- Reuti > I'd appreciate anyone sharing notes/information about > creating 'a forced complex' and using 'nvidia-smi to change > all GPU to compute exclusive'. > > Thanks for your time and help with this. > > Best, > g > > -- > Gowtham > Advanced IT Research Support > Michigan Technological University > > (906) 487/3593 > > _______________________________________________ > users mailing list > [email protected] > https://gridengine.org/mailman/listinfo/users _______________________________________________ users mailing list [email protected] https://gridengine.org/mailman/listinfo/users
