One of the clusters we have at Michigan Tech has 3 compute nodes, each with 2 nVidia GPU cards (GeForce GTX 260) which in turn have 216 CUDA cores each. Each of these 3 compute nodes have 4 CPUs.
I followed Jon Forrest's instructions here: http://marc.info/?l=npaci-rocks-discussion&m=129668495229097&w=2 I got it work with devdriver_4.0_linux_64_270.41.19.run cudatoolkit_4.0.17_linux_64_rhel5.5.run gpucomputingsdk_4.0.17_linux.run with very little modifications to the instructions in the above URL. After setting the necessary variables in user's .bashrc, I could compile the GPU Computing SDK and when run deviceQuery on compute nodes (either by logging into compute node & running it OR submitting a simple SGE job) and get desired, meaningful results about both GPU cards: ... Found 2 CUDA Capable device(s) Device 0: "GeForce GTX 260" CUDA Driver Version / Runtime Version 4.0 / 4.0 CUDA Capability Major/Minor version number: 1.3 ... Device 1: "GeForce GTX 260" CUDA Driver Version / Runtime Version 4.0 / 4.0 CUDA Capability Major/Minor version number: 1.3 ... deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 4.0, CUDA Runtime Version = 4.0, NumDevs = 2, Device = GeForce GTX 260, Device = GeForce GTX 260 ... I followed that chain of emails in the above URL and from Dung Do's note: http://marc.info/?l=npaci-rocks-discussion&m=129675227702026&w=2 I understand that it's possible to control usage of GPU cores from SGE. I have done the following a few times before - creating a new queue - creating host list and assigning it to a queue but what I have not done are - create a forced complex (cudaonly) - use nvidia-smi to change all GPU to compute exclusive Following that email further, I too would like to just use 3 CPU cores from each compute nodes leaving 1 CPU core for non-GPU jobs. I'd appreciate anyone sharing notes/information about creating 'a forced complex' and using 'nvidia-smi to change all GPU to compute exclusive'. Thanks for your time and help with this. Best, g -- Gowtham Advanced IT Research Support Michigan Technological University (906) 487/3593 _______________________________________________ users mailing list [email protected] https://gridengine.org/mailman/listinfo/users
